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PREFACE 

This  report  presents  the  results  of  efforts  of  the  Range  Commanders  Council  (RCC) 
Signature  Measurement  Standards  Group  (SMSG)  for  completion  of  Task  SMSG-18  and  the 
publication  of  “ Development  of  Electro -optical  (EO)  Standard  Processes  for  Application .”  This 
document  defines  the  process  of  characterizing  the  measurement  capabilities  of  EO  measurement 
systems,  and  establishes  a  demonstration  program  that  verifies  and  quantifies  the  improvement  in 
measurement  capability  that  will  result  from  implementing  documented  processes.  The  standard 
processes  include  calibration  procedures,  standard  data  reduction  procedures  to  transform  raw  data 
to  standard  engineering  unit  data,  standard  terminology,  and  standard  units  of  measure. 

For  development  of  this  standard,  the  RCC  gives  special  recognition  to: 

Task  Lead:  Mr.  Jeffrey  Burks:  Associate  Representative,  SMSG 
Air  Force  Research  Laboratory  (AFRL/RYS) 


For  their  exceptional  technical  support  during  the  developmental  effort,  the  RCC  is  also 
indebted  to  the  following  individuals: 

•  Mr.  Chris  Dobbins:  Chair,  SMSG 

U.S.  Army  Aviation  and  Missile  Research  Development  and  Engineering 

•  Mr.  Roger  W.  Davis:  Secretary  SMSG 

NAVAIR  Patuxent  River,  Dyncorp  Range  Technical  Services 

•  Ms.  Kathryn  Dietz:  Associate  Representative,  SMSG 
Centers  for  Countermeasures 


Please  address  any  questions  to: 

Secretariat,  Range  Commanders  Council 
ATTN:  TEDT-WS-RCC 
1510  Headquarters  Avenue 

White  Sands  Missile  Range,  New  Mexico  88002-5110 
Phone:(575)  678-1107  "  DSN  258-1107 
Fax:  (575)  678-9517  DSN  258-9517 
Email:  usarmv.wsmr.atec.list.rcc@mail.mil 
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CHAPTER  1 

BACKGROUND 


1.1  Summary 

This  report  presents  the  results  of  an  effort  by  the  Signature  Measurement  Standards 
Group  (SMSG)  of  the  Range  Commanders  Council  (RCC)  to  define  and  document  standard 
processes  to  apply  electro-optical  (EO)  measurement  systems.  The  over- arching  objective  of  this 
task,  Task  SMSG- 19,  as  well  as  other  yet  to-be-defined  tasks,  is  to  develop  an  accreditation 
program  for  laboratories  involved  in  collecting  ultraviolet  through  infrared  (IR)  radiometric 
measurements. 

In  this  document,  the  term  electro-optical/infrared  (EO/IR)  is  used  to  identify  the 
community  involved  with  collecting  ultraviolet,  visible,  and  infrared  radiometric  measurements 
in  the  electromagnet  spectral  region  spanning  0.2  pm  to  25  pm. 

1.2  Approach  for  Developing  the  Accreditation  Program 

The  envisioned  accreditation  program  is  being  developed  along  lines  similar  to  those 
followed  by  the  radar  cross  section  (RCS)  community  (Reference  1.11a).  Part  of  the  effort 
consists  of  modifying  the  evaluation  criteria  developed  for  the  RCS  certification  program,  which 
is  based  on  the  American  National  Standards  Institute  (ANSI  Z540.1-1994,  R2002) 
criteria  (see  Reference  1.11b).  With  modifications,  the  RCS  certification  program  and  the  ANSI 
evaluation  criteria  were  found  to  generally  suit  the  EO/IR  community.  While  the  RCS 
certification  program  is  based  on  the  aforementioned  ANSI  standard,  the  SMSG  recommends 
that  future  work  focus  on  the  design  of  an  accreditation  program  based  on  ISO/IEC- 17025 
(Reference  1.11c).  The  17025  standard,  which  is  part  of  the  ISO-9000  series  and  is  more 
familiar  to  industry,  replaced  ISO  Guide  25,  which  was  the  international  equivalent  to 
ANSI  Z-540. 

This  effort  defined,  documented,  and  tested  processes  that  can  be  standardized  and 
applied  to  EO/IR  measurement  systems.  The  standard  processes  drafted  during  this  effort  define 
levels  of  characterization  that  are  synonymous  with  levels  of  fidelity  and  the  processes  to  meet 
those  levels.  The  levels  of  characterization  allow  cost  effective  options  to  meet  test  requirements 
and  to  allow  documentation  of  spiral  improvement  of  capability.  The  standard  processes  include 
calibration  procedures,  standard  data  reduction  procedures  to  transform  raw  data  to  standard 
engineering  unit  data,  standard  terminology,  and  standard  units  of  measure. 

1.3  Quality  Documentation  Book  (The  Lab  Book ) 

Employing  material  developed  under  this  task  and  using  the  aforementioned  RCS 
certification  program  as  a  guide,  the  Optical  Measurements  Facility  (OMF)  at  the  Air  Force 
Research  Laboratory  (AFRL),  Wright  Patterson  Air  Force  Base  (AFB),  developed  a  Quality 
Documentation  Book.  This  book,  commonly  referred  to  as  the  Lab  Book,  serves  as  a 
demonstration  of  the  processes  required  for  accreditation  of  the  EO/IR  measurement  capabilities. 
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The  OMF  conducted  a  self-assessment  and  the  results  were  independently  evaluated  by  two 
outside  organizations. 

The  evaluation  criteria  applied  during  the  OMF  assessments,  presented  in  Chapter  2, 
were  subsequently  modified  to  eliminate  redundancy  and  ambiguity,  and  to  make  the  criteria 
easier  to  interpret.  The  modified  criteria,  provided  in  Chapter  3,  reflect  lessons  learned  from 
three  assessments  by  different  groups;  however,  they  were  applied  to  the  same  AFRL  Lab  Book. 
One  recommendation  resulting  from  this  effort  is  that  the  evaluation  criteria  should  be  tested 
more  thoroughly  and  broadly  at  facilities  having  significantly  different  functions  and  capabilities. 

During  the  course  of  the  Task  SMSG-18  effort,  a  survey  was  designed,  developed, 
disseminated,  collated,  and  analyzed.  The  survey  form  used  and  the  subsequent  results  and 
analyses  are  presented  at  Chapter  4  and  Chapter  5,  respectively. 

The  draft  standard  for  characterization  of  EO/IR  measurement  facilities  and  a  draft 
handbook  for  conducting  EO/IR  measurements  are  at  Chapter  6  and  Chapter  7,  respectively.  The 
information  in  Chapter  6  and  Chapter  7,  plus  the  work  product  from  Task  SMSG-19  (RCC 
document  809-10,  Standards  and  Procedures  for  Application  of  Radiometric  Sensors, 

Reference  El  Id),  should  be  considered  by  the  broader  EO/IR  measurements  community  as 
foundations  (however  imperfect)  for  a  full-fledged  accreditation  program. 


The  work  presented  in  this  report  is  not  intended  to  be  a  final  product  that 
represents  the  full  spectrum  of  references  needed  for  an  accreditation 
program  that  would  be  consistent  with  best  practices  and  those  documented 
by  the  ISO.  Rather,  the  material  presented  here  is  intended  to  be  the  basis 
for  further  review,  dialog,  and  refinement  by  the  EO/IR  measurements 
community. 


1.4  Task  Overview 

The  original  scope  of  the  effort  covered  by  Task  SMSG-18  addressed  the  need  to  define 
and  document  standard  processes  for  electro-optical  (EO)  measurement  systems.1  Ultimately, 
the  efforts  undertaken  by  this  task  would  be  applied  to  the  establishment  of  an  accreditation 
program  for  EO/IR  measurement  capabilities. 

The  precedence  for  developing  an  accreditation  program  for  EO/IR  measurement 
capabilities  lies  in  another  effort  by  the  SMSG  between  1999  and  2001  that  developed  a 
certification  process  and  standard  for  the  RCS  community  (Reference  1.11a).  That  certification 
process  was  based  on  an  ANSI  standard  that  applies  to  measurements  made  by  calibration 
laboratories  (Reference  1.11b).  The  ANSI  standard  was  the  United  States  (U.S.)  equivalent  to  an 
ISO  guide  that  was  subsequently  replaced  by  a  newer  standard  (Reference  1.11c). 


1  In  this  document,  the  terms  electro-optical  (EO)  and  electro-optical/infrared  (EO/IR)  are  used  interchangeably  and 
generically  are  used  to  identify  the  community  involved  with  collecting  ultraviolet,  visible,  and  infrared  radiometric 
measurements  in  the  electromagnet  spectral  region  spanning  0.2  pm  to  25  pm. 
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Some  members  of  the  SMSG  familiar  with  both  the  RCS  and  the  EO/IR  measurements 
community  recommended  that  the  EO/IR  community  follow  an  approach  similar  to  the  one  used 
by  the  RCS.  While  the  RCS  measurement  community  is  diverse,  and  the  demonstration  program 
that  was  developed  and  executed  under  SMSG  auspices  addressed  some  of  that  diversity,  the 
EO/IR  community  is  even  more  diverse.  Unlike  RCS  measurements,  which  are  overwhelmingly 
driven  by  the  military,  EO/IR  measurements  have  a  much  wider  base  of  practitioners  and 
consumers  than  just  the  military.  This  wider  base  includes  the  military  and  civil  agencies  like 
the  National  Aeronautics  and  Space  Administration  (NASA),  the  National  Institute  of  Standards 
and  Technology  (NIST),  the  Department  of  Agriculture,  as  well  as  many  industries  and  everyday 
consumer  products. 

The  Task  SMSG- 18  products  documented  in  this  report  are  part  of  a  larger  multi-year 
evolutionary  effort  that  includes  Task  SMSG- 19  (see  Reference  1.1  Id),  as  well  as  tasks  yet  to  be 
more  fully  defined  and  executed.  Because  of  the  aforementioned  diversity  of  the  EO/IR 
measurements  community,  the  level  of  effort  to  achieve  consensus  by  the  EO/IR  measurements 
community  is  significantly  greater  than  that  of  the  RCS  measurements  community.  For  example, 
clarification  is  even  required  for  some  of  the  most  basic  terminology  such  as  certification  and 
accreditation.2 

One  goal  of  Task  SMSG- 18  was  to  establish  standardized  processes  that  would  define 
levels  of  characterization  that  are  synonymous  with  levels  of  fidelity  and  the  processes  needed  to 
meet  those  characterization  levels.  By  establishing  and  permitting  different  levels  of 
characterization,  cost  effective  options  to  meet  test  requirements  would  allow  for  the 
documentation  of  spiral  improvements  in  capabilities  at  any  given  measurement  facility.  The 
standard  processes  to  be  defined  included  calibration  procedures,  standard  data  reduction 
procedures  to  transform  raw  data  to  standard  engineering  unit  data,  and  standard  terminology  and 
units  of  measure. 


On  the  above  point  regarding  standard  terminology  and  units  of  measure, 
the  terminology  and  units  expressed  herein  are  not  as  expansive  as  those 
found  in  Reference  1.1  Id  (from  Task  SMSG-19). 


2  Since  the  early  1990s,  there  has  been  broad  confusion  with  respect  to  the  terms  accreditation  and  certification  as 
they  may  be  applied  to  formal  programs  that  seek  to  document  compliance  with  standards.  The  long-running 
confusion  was  addressed  by  the  International  Organization  for  Standards  (ISO)  in  1996  when  it  published  ISO/IEC 
Guide  2:1 996,  which  sets  out  the  definitions  specifically  related  to  standardization,  certification,  and  laboratory 
accreditation,  and  again  in  December  2002  with  a  special  memo  on  the  topic  (See  Reference  4).  Simply 
summarized,  accreditation  applies  to  the  competence  of  organizations  or  persons  while  certification  applies  to 
products,  processes,  or  services.  Even  with  these  clarifications,  issues  still  arise.  For  accreditation,  an  authoritative 
body  gives  formal  recognition  that  a  body  (organization)  or  person  is  competent  to  carry  out  specific  tasks.  For 
certification,  a  third  party  gives  written  assurance  that  a  product,  process,  or  service  conforms  to  specified 
requirements.  In  RCC  804-01,  the  term  certification  is  used  and  a  third  party  (  not  an  authoritative  body )  gives 
written  assurance  that  a  body  (not  a  product,  process,  or  sendee)  is  competent  to  perform  radar  cross  section 
measurements  in  the  context  of  the  facility  employed.  In  this  document,  the  term  accreditation  is  preferred  because 
the  procedures  envisioned  by  the  authors  of  this  proposed  standard  more  closely  align  with  the  ISO  definition  for 
that  term. 
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The  efforts  undertaken  through  Task  SMSG-18  resulted  in  several  draft/baseline 
documents  that  will  support  the  establishment  of  an  accreditation  program  for  laboratories 
involved  in  collecting  ultraviolet  through  infrared  radiometric  measurements.  Given  the  many 
challenges  encountered,  such  as  a  funding  reduction  and  a  disruption  due  to  contractual  issues, 
there  was  still  a  great  deal  of  work  accomplished  under  this  task.  The  challenges,  coupled  with 
the  large  size  and  scope  of  establishing  a  fully  vetted  accreditation  program,  means  much  work 
remains  to  be  done.  Because  of  the  diversity,  and  the  relatively  small  number  of  participants  in 
this  task,  this  document  should  not  be  construed  as  the  final  word  on  accrediting  facilities  or 
organizations  engaged  in  EO/IR  measurements.  Instead,  this  document  provides  some  of  the 
basic  building  blocks  of  an  accreditation  program. 

Although  the  material  presented  herein  may  appear  to  be  relatively  complete,  a  broad 
community  effort  is  needed  to  verify  and  validate  the  ideas  presented.  Therefore,  significant 
changes  are  anticipated,  but  because  the  application  areas  are  so  broad  and  complex,  establishing 
a  single  process  with  a  single  set  of  evaluation  criteria  may  not  be  possible. 

1.4.1  Actions  Taken  to  Address  the  SMSG-18  Requirements.  In  summary,  the  SMSG-18  goals 
were  to  address  the  task  requirements  by: 

a.  Establishing  or  adopting  baseline  evaluation  criteria  (Chapter  2). 

b.  Drafting  criteria  for  EO  measurement  capabilities  evaluation  consistent  with 
published  standards  for  measurement  facilities  (Chapter  3). 

c.  Developing  a  survey  (Chapter  4). 

d.  Distributing  the  survey  and  collecting  results. 

e.  Compiling  and  presenting  the  survey  results  (Chapter  5). 

f.  Drafting  a  standard  for  EO  measurement  process  (Chapter  6). 

g.  Drafting  a  handbook  for  characterizing  the  measurement  capabilities  of  EO/IR 
measurement  systems  (Chapter  7). 

h.  Production  of  this  final  report. 

When  task  SMSG-18  began  in  earnest,  the  task  lead  contracted  for  support.  During  the 
first  year  of  support,  the  contractor  underwent  both  organizational  and  personnel  changes, 
causing  some  diversions  and  realignments  of  tasks  and  priorities.  Nevertheless,  much  work  was 
accomplished  toward  accomplishing  the  above  stated  goals.  The  task  was  originally  planned  as  a 
two-year-plus  funded  project  under  the  RCC  Omnibus  Contract,  funding  in  the  third  year  was 
reduced  to  zero.  Those  factors  caused  the  SMSG  to  reassess  and  to  some  extent  adjust  the 
methods  and  means  to  produce  products  that  would  be  useful  to  the  broader  EO/IR  community. 

The  RCC  SMSG  selected  the  AFRL/RX  OMF  as  a  pilot  facility  for  developing  and 
testing  some  of  the  elements  for  an  accreditation  program  for  measurements  conducted  in  the 
EO/IR  spectral  region.  Lessons  learned  during  the  pilot  effort  were  used  to  modify  the  draft 
EO/IR  accreditation  program  materials  presented  in  this  report. 

The  purpose  of  the  OMF  is  to  measure,  correlate,  and  analyze  the  optical  behavior  of 
materials.  The  OMF  instruments  operate  in  the  ultraviolet  through  the  infrared  wavelength 
region. 
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1.4.2  Core  QMF  Optical  Measurement  Instruments.  The  core  OMF  optical  measurement 
instruments  include: 

a.  Ultraviolet-visible-near-infrared  spectrometer  with  directional  hemispherical 
reflectance  attachments. 

b.  Infrared  spectrometer  with  directional-hemispherical  reflectance  and  directional- 
emittance  attachments. 

c.  Laser  and  broadband  bidirectional-reflectance  scatterometers. 

The  OMF  prepared  a  draft  Lab  Book  using  the  ANSI  Z540. 1-1994  (R2002)  standard  that 
documents  the  measurement  procedures,  instrument  error  characterization,  and  quality  control 
procedures  used  at  the  facility.  This  same  standard  has  been  used  by  the  RCC  SMSG  for  the 
RCS  Range  Accreditation  Program. 

The  focus  of  this  report  is  on  the  evaluation  process  and  the  establishment  of  evaluation 
criteria.  Both  the  independent  assessments  and  the  self-assessment  show  clearly  that  a 
significant  amount  of  work  remains  to  be  done  before  the  OMF  could  achieve  full  compliance. 
Though  achieving  a  great  deal  through  this  exercise,  becoming  fully  compliant  is  not  a  current 
objective  for  the  OMF.  However,  full  compliance  may  become  a  goal  as  the  accreditation 
program  matures  and  the  OMF,  like  other  facilities/organizations,  becomes  more  familiar  with 
the  overall  program. 

A  good  framework  has  been  developed,  but  further  development  is  needed  before 
undergoing  a  thorough  compliance  review.  The  management  of  the  OMF  has  determined  that 
they  are  not  seeking  formal  accreditation,  but  they  are  interested  in  developing  a  better- 
documented  facility  calibration  and  quality  control  process.  Such  a  process  will  give  assurance 
to  their  customers  that  sound  measurement  practices  and  error  assessments  are  being  performed. 
A  reasonable  foundation  for  developing  and  implementing  a  quality  assurance  and  control  plan 
for  their  laboratory  is  provided  by  ANSI-Z-540. 1-1994  (R2002). 

The  evaluations  conducted  on  the  OMF  represented  an  initial  look  at  the  OMF  processes 
from  the  perspective  of  ANSI-Z-540. 1-1994  (R2002).  The  RCC/SMSG  had  two  primary 
objectives  for  this  evaluation.  They  were: 

a.  Perform  a  quality  system  assessment. 

b.  Strictly  follow  (as  much  as  feasible)  the  certification/accreditation  process  and 
evaluation  criteria  to  provide  lessons  learned  and  opportunities  for  improvement  on 
future  projects. 

1.5  Evolution  of  the  Evaluation  Criteria 

The  OMF  conducted  a  self-assessment  by  using  their  OMF  Lab  Book.  Additionally,  two 
independent  assessments  were  conducted.  Evaluation  of  the  Lab  Book  followed  the  evaluation 
criteria  given  in  Chapter  2,  which  were  virtually  identical  to  those  developed  by  the  SMSG  for 
the  standard  that  has  been  applied  to  RCS  measurement  facilities.  The  criteria  employed  for 
assessing  the  OMF  were  subsequently  modified  based  on  lessons-learned  during  the  three 
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assessments.  The  modified  criteria  form  the  final  recommended  criteria  under  this  effort  and  are 
provided  in  Chapter  3.  The  modifications  were  suggested  to  remove  ambiguous,  duplicate,  or 
irrelevant  requirements,  and  to  make  some  of  the  language  more  readable.  The  evaluators 
collectively  recommended  further  review  of  the  criteria  presented  in  Chapter  3  by  a  broader 
collection  of  EO/IR  subject  matter  experts  before  final  adoption  as  part  of  an  accreditation 
program.  Even  if  the  EO/IR  community  elects  to  follow  a  path  defined  by  ISO/IEC  17025,  the 
modified  criteria  should  be  considered  as  a  starting  point.  (See  paragraph  1.10,  Conclusions  and 
Recommendations  for  more  details.) 

As  was  implemented  by  a  substantial  number  of  RCS  measurement  facilities,  the  OMF 
Lab  Book  is  web-based  and  interactive  in  nature,  with  hyperlinks  to  relevant  sections  and 
reference  materials.  All  of  the  links  tested  worked  and  linked  the  evaluator  to  the  correct 
content.  No  on-site  review  was  performed,  so  none  of  the  on-site  review  criteria  was  evaluated. 

As  has  proven  to  be  typical  in  the  RCS  measurement  facilities  certification  program,  the 
initial  evaluation  of  the  OMF  showed  that  the  facility  made  good  progress,  but  had  not  achieved 
passing  scores.  That  result  is  actually  the  norm.  Because  it  was  recognized  in  the  RCS  program 
that  facilities  never  managed  to  achieve  a  passing  score  at  the  mid-term  evaluation,  the  process 
evolved  such  that  the  communications  between  the  evaluators  and  the  facility  are  best  conduced 
on  a  continuous  basis  so  that  improvements  can  be  made  and  changes  implemented  as 
weaknesses  are  uncovered.  As  a  minimum,  after  the  highest  priority  sections  are  reviewed,  a 
mid-term  report  is  issued  to  the  facility  so  that  it  may  begin  tackling  those  criteria  found  to  be 
less  than  satisfactory.  The  recommended  practice  is  that  the  evaluators  hold  regularly  scheduled 
teleconferences  with  the  facility  during  the  review  process.  The  teleconference  helps  to  prevent 
surprises,  provides  the  facility/organization  with  an  opportunity  to  clarify  requirements,  and 
gives  an  opportunity  to  clarify  the  methodologies  for  meeting  requirements. 

An  important  goal  of  Task  SMSG-18  is  wide  dissemination  of  this  final  report. 

Following  the  precedence  set  by  the  RCS  community,  and  as  is  typically  followed  in  the  ISO 
community,  the  results  of  evaluations  are  confidential  in  nature.  Generally,  the  intimate  details 
found  during  such  evaluations  are  not  reported  (e.g.,  scores  or  specific  comments  about  how 
individual  criterion  were  (or  were  not)  addressed).  An  evaluated  organization  may  elect  to 
publicly  share  its  evaluation  results.  As  the  OMF  is  a  U.S.  government  facility  and  various 
public -release  restrictions  apply  to  information  about  such  facilities,  we  present  in  Chapter  2 
only  the  criteria  and  some  of  notes  that  were  provided  by  the  evaluators,  but  only  to  the  extent 
such  notes  illuminate  issues  that  other  facilities  might  encounter. 

1.6  Recommended  Evaluation  Criteria 

The  ANSI  Z-540. 1-1994  (R2002)  standard  and  the  international  equivalent  (ISO 
Guide  25)  have  been  superseded  by  ISO/IEC- 17025  and  its  U.S.  equivalent,  ANSFNCSF 
Z540. 3-2006.  ISO/IEC- 17025  differs  significantly  in  organization,  but  contains  very  similar 
quality  management  techniques  and  criteria.  Changes  were  made  to  better  align  the  newer 
standard  with  other  ISO  quality  systems  standards  and  guidance.  The  primary  reason  the  OMF 
elected  to  use  the  outdated  ANSI  Z540.1-1994  (R2002)  standard  is  due  to  the  availability  of  the 
evaluation  criteria  developed  for  it  by  the  DoD  RCS  Certification  community.  The  National 
Conference  of  Standards  Laboratories  (NCSL)  Handbook  for  ANSI  Z540.1-1994  contains 
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suggested  evaluation  factors  written  in  the  form  of  declarative  statements  that  are  very  broad  in 
scope.  The  RCS  community  rewrote  those  relevant  evaluation  factors  in  an  interrogative  format 
that  was  customized  for  the  RCS  range  environment.  The  OMF  used  the  DoD  RCS  Certification 
criteria  as  the  basis  for  their  own  Optical  Properties  Laboratory  Book  and  evaluation  criteria  by 
adapting  them  to  the  EO/IR  laboratory  setting. 

With  some  effort,  a  similar  set  of  evaluation  criteria  could  be  written  for  ISO- 17025. 

This  could  be  beneficial  in  the  future,  since  ISO- 17025,  and  the  ISO-9000  series  of  which  it  is  a 
part,  are  more  familiar  to  industry  and  could  add  to  the  prestige  and  economics  of  certification. 
For  the  present,  the  quality  system  described  in  ANSI  Z540. 1-1994  (R2002)  meets  the  needs  of 
both  the  RCS  and  the  EO/IR  accreditation  programs.  Ultimately,  the  recommendation  is  for  the 
EO/IR  community  to  employ  ISO/IEC- 17025. 

1.7  Survey 

The  contractor  supporting  Task  SMSG-18  distributed  the  first  draft  of  what  was  called 
the  Development  of  EO  Standard  Processes  for  Application  (EOSPA)  with  survey/questionnaire 
to  the  organizations  identified  by  the  RCC  to  solicit  feedback.  The  survey/questionnaire 
included  questions  that  were  intended  to  help  identify  an  initial  set  of  ranges  that  would  be 
willing  to  demonstrate  the  standard  procedures  to  be  documented  in  the  EOSPA  draft.  Eleven 
surveys  were  completed  and  collected  after  three  rounds  of  survey  distributions. 

The  first  version  of  the  draft  of  EOSPA  in  the  format  of  a  Microsoft  Word®  document 
with  survey  and  instructions  attached  was  distributed  to  about  10  to  15  attendees  during  an 
SMSG  meeting  held  in  October  17-18,  2006.  Unfortunately,  none  of  the  organizations  surveyed 
sent  back  feedback  by  the  deadline.  To  make  responding  to  the  survey  easier,  a  second  version 
in  the  form  of  an  Microsoft  Excel®  spreadsheet  was  completed  on  November  27,  2006.  The 
survey  was  distributed  again  to  the  known  EO/IR  ranges  and  SMSG  members.  Feedback  from 
four  groups  was  collected  in  the  second  round  but  none  of  them  included  sample  procedures  or 
reports.  One  of  the  respondents  was  not  willing  to  share  its  responses  with  the  broader 
community,  so  those  responses  were  not  included  in  the  tally.  To  solicit  more  feedbacks,  the 
SMSG  modified  and  redistributed  the  survey  via  e-mail  during  January  10-11,  2007  to  about  1 10 
contacts  involved  in  EO/IR  measurements.  Follow-up  telephone  calls  were  made  to  all  the 
contacts.  Feedback  from  seven  groups  was  collected  but  none  of  them  included  sample 
procedures  or  reports.  Finally,  in  an  effort  to  collect  sample  procedures  and  reports,  e-mail 
requests  and  phone  calls  were  made  during  March  8-19,  2007  to  the  organizations  who 
responded  to  the  survey.  This  effort  still  did  not  result  in  any  sample  procedures  or  reports. 

Some  organizations  informed  the  contractor  that  the  procedures  and  reports  were  for  their 
internal  use  only.  Their  management  did  not  approve  the  request  for  distribution.  Other 
organizations  simply  said  that  they  did  not  have  any. 


3  Following  task  initiation,  and  subsequent  to  some  contractual  issues  being  resolved,  the  contractor  that  was 
awarded  the  task  to  support  AFRL  on  Task  SMSG-18,  Mission  Research  Corporation,  was  subsequently  bought  by 
Alliant  TechSystems  Inc.,  now  known  as  ATK. 
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Despite  these  challenges,  the  survey  did  result  in  the  collection  of  a  significant  amount  of 
information,  particularly  as  relates  to  instrumentation.  The  survey  is  provided  as  Chapter  4,  with 
the  results  provided,  without  attribution  to  the  participants,  in  Chapter  5. 

1.8  Draft  Documentation  Standard 

Though  the  effort  to  collect  information  from  the  community  proved  to  be  less  successful 
than  was  desired,  the  contractor  developed  and  delivered  a  draft  document,  Electro-Optical 
Standard  for  Applications,  in  October  2006. 

That  document  provided  a  draft  standard  for  characterizing  the  measurement  capabilities 
of  EO/IR  measurement  systems  in  the  hope  that  it  would  assist  in  the  establishment  of  a 
demonstration  program  to  verify  and  quantify  the  improvement  in  measurement  capabilities  that 
would  result  from  implementing  documented  standards.  The  draft  standard  set  out  to  define 
levels  of  information  content  that  are  synonymous  with  levels  of  knowledge  and  expertise  and 
the  requirements  needed  to  meet  those  levels.  For  example,  the  lowest  level  would  be  capable  of 
providing  qualitative  information.  A  middle  level  might  provide  relative  quantitative 
information.  The  highest  level  would  incorporate  both  lower  levels  and  would  be  able  to  provide 
absolute  quantitative  information  with  specified  measurement  uncertainties.  The  levels  allowed 
were  intended  to  specify  cost  effective  options  to  meet  preset  test  requirements. 

The  document,  presented  in  Chapter  6,  Draft  Document  Standards  for  Characterizing  the 
Measurement  Capabilities  of  EO/IR  Measurement  Systems,  provides  a  draft  standard  for 
planning,  executing,  and  reporting  signature  measurements.  It  also  describes  the  resources 
available  to  the  EO/IR  range  community  and  recommends  procedures  to  help  testers  make 
efficient,  effective  use  of  instrumentation.  Included  are: 

a.  EO  instrument  characterization  and  calibration  procedures. 

b.  Standard  data  reduction  procedures  to  transform  raw  data  to  engineering  units. 

c.  Measurement  uncertainty  analysis. 

d.  Standard  terminology  and  units  of  measure  (and  see  Reference  1.1  Id). 

As  with  the  modified  evaluation  criteria,  the  draft  standard  is  intended  to  be  a  starting 
point  for  discussion  and  consensus  building  across  the  EO/IR  measurements  community. 

1.9  Draft  Handbook  for  Electro-optical  Measurements 

Out  of  the  various  sub-tasks  that  comprised  Task  SMSG-18,  a  draft  handbook  was  born. 
The  draft  handbook  was  based,  in  part,  on  RCC  804-01  (Reference  l.lla)  and  IRIG  Standard 
RCC  802-98,  Tactical  Missile  Signatures  Measurement  Standard  and  Definitions 
(Reference  l.lli).  The  draft  handbook,  presented  in  Chapter  7,  is  intended  to  assist  SMSG 
members  and  others  engaged  in  electro-optical  and  infrared  signature  measurements  by 
documenting  standard  procedures.  The  overall  structure  and  language  used  in  the  draft 
handbook  parallels  that  of  the  Handbook  for  the  Interpretation  and  Application  of  ANSI/ 

NCSL  Z540- 1-1994. 
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As  with  the  modified  evaluation  criteria  and  the  draft  standard,  the  draft  handbook  is 
intended  to  be  a  part  of  the  discussion  and  consensus  building  across  the  EO/IR  measurements 
community  as  it  moves  forward  in  developing  an  accreditation  program. 

1.10  Conclusions  and  Recommendations 

Although  completing  Task  SMSG-18  was  difficult,  the  resulting  reference  material  was 
substantial.  That  material,  much  of  which  is  collected  in  this  final  report,  can  serve  as  baseline 
documents,  or  as  points  of  departure,  for  an  expanded,  robust  dialog  among  EO/IR  measurement 
community  members.  The  desire  is  that  the  fruits  of  the  long  and  arduous  task  of  completing 
Task  SMSG-18  support  a  community-wide  accreditation  program,  though  much  collaborative 
work  remains  to  be  accomplished  to  achieve  that  end. 

The  OMF  made  significant  progress  toward  reaching  their  goal  of  having  an  ANSI 
Z540. 1-1994  (R2002)-compliant  calibration  and  quality  control  process.  This  evaluation  has 
identified  the  specific  areas  where  the  OMF  needs  to  improve  or  develop  processes  in  order  to 
achieve  that  goal.  They  developed  a  comprehensive  web-based  laboratory  book  that  is 
interactive  in  nature,  has  hyper-links  to  relevant  sections  and  reference  materials,  and  is  a  good 
example  for  other  EO/IR  laboratories.  A  great  deal  of  effort  has  gone  into  developing  their 
processes  and  we  recommend  that  the  OMF  continue  to  improve  their  Laboratory  Book  to  the 
point  it  can  be  submitted  for  formal  compliance  review. 

The  recommendation  of  the  participants  is  that  evaluation  criteria  based  on  either 
ANSI/NCSL  Z540. 3-2006  or  the  ISO- 17025  be  adopted  by  the  SMSG  and  RCC  for  the  EO/IR 
laboratory  community.  Alternatively,  and  as  a  minimum,  evaluation  criteria  based  on  ANSI 
Z540. 1-1994  (R2002),  as  presented  in  Chapter  3,  could  be  used,  but  because  of  the  breadth  and 
diversity  of  facilities/organizations  that  might  follow  this  program,  the  more  contemporary 
standards  would  likely  be  more  palatable. 

The  recommendation  of  the  participants  is  that  the  evaluation  criteria  in  Chapter  3  be 
reviewed  and  additional  testing  of  those  criteria  by  multiple  facilities  occurs  before  finalizing  the 
criteria.  A  multi-facility  test  of  the  process  and  criteria  would  follow  the  same  path  as  the  RCS 
certification  program,  in  which  three  very  different  facilities  were  employed  to  test  the  process 
and  criteria.  After  more  broadly  tested  criteria  have  been  refined,  they  should  be  presented  to  the 
EO/IR  community  in  a  document  including  clearly  written  definitions  of  the  criteria.  In  order  to 
reach  a  broad  consensus,  the  document  should  also  include  the  background  of  how  the  criteria 
were  evolved. 

The  participants  further  recommend  that  the  percentages  of  Satisfactory,  Needs 
Improvement,  and  Unsatisfactory  be  more  widely  assessed  to  finalize  how  they  will  be  applied, 
especially  given  the  recommendation  that  there  be  different  levels  of  measurement 
characterization  (see  Chapter  7,  Draft  Handbook  for  Electro-Optical  Measurements). 

Finally,  this  report  has  not  addressed  the  administrative  procedures  (such  as  what  entity 
will  administer  the  program,  the  initial  application  for  review,  reviewer’s  qualifications,  and 
panel  appointment)  for  a  formal  accreditation  program.  We  recommend  the  SMSG  undertake  a 
task  to  establish  those  elements  of  the  accreditation  program. 
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CHAPTER  2 


EVALUATION  CRITERIA  APPLIED  TO  THE  OPTICAL  MEASUREMENTS 

FACILITY  (OMF)  “LAB  BOOK” 

2.1  Priority  Ordering  of  Evaluation  Criteria 

The  evaluation  criteria  are  presented  below  in  sections  grouped  in  priority  order,  not  in 
numerical  order.  The  criteria  are  prioritized  as  follows: 

Priority  1.  Those  elements  of  a  measurements  quality  program  having  direct  bearing  on 
the  quality  of  the  measurements  being  taken  and  reported. 

Priority  2.  Those  elements  of  a  measurements  quality  program  having  indirect  bearing 
on  the  quality  of  the  measurements  being  taken  and  reported. 

Priority  3.  Administrative  elements  of  a  measurements  quality  program,  which 

do  not  directly  nor  indirectly  impact  the  quality  of  the  measurements  taken 
and  reported,  but  which  do  reflect  good  management  practices  that,  in-tum, 
enhance  the  overall  quality  measurement  program. 


2.2  Criteria  Assessments 

The  following  assessments  are  as  follows: 

S  Satisfactory  if  the  requirement  is  met  fully. 

N  Needs  Improvement  if  the  requirement  is  met  to  some  reasonable  degree. 

U  Unsatisfactory  if  the  requirement  is  barely  met  or  not  met  at  all. 

N/A  Not  Applicable  if  the  requirement  is  not  relevant  to  the  facility  under  evaluation. 


NOTE 


The  OMF  self-assessment  was  rated  as  either  being  met  satisfactorily 
(a  check  mark)  or  deficient  (D).  This  approach  may  suffice  for  an  initial 
internal  assessment,  but  may  yield  surprising  results  when  an  external 
reviewer  utilizes  a  broader  assessment  scale. 
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To  determine  an  overall  score  by  priority,  the  scheme  in  Table  2-1  was  applied. 


TABLE  2-1.  COMPOSITION  OF  PASSING  SCORES 

Priority 

Evaluation 

Criteria 

Notes 

1 

S  >  85% 

To  achieve  accreditation,  85%  or  more  of  the  Priority  1  criteria  must 
be  scored  Satisfactory,  with  the  rest  being  scored  Needs 

N<  15% 

Improvement. 

No  Unsatisfactory  (and  no  Not  Applicable)  scores  are  permitted  for 
Priority  1  Sections. 

o 

ll 

£ 

2 

S  >  75% 

To  achieve  accreditation,  75%  or  more  of  the  applicable*  Priority  2 
criteria  must  be  scored  Satisfactory;  no  more  than  10%  of  the  criteria 
can  be  rated  Unsatisfactory  (with  the  balance  being  Needs 
Improvement). 

N  <  Balance 

U<  10% 

3 

S  >  75% 

To  achieve  accreditation,  75%  or  more  of  the  applicable*  Priority  3 
criteria  must  be  scored  Satisfactory;  no  more  than  10%  of  the  criteria 
can  be  rated  Unsatisfactory  (with  the  balance  being  Needs 
Improvement). 

N  <  Balance 

U<  10% 

*  Some  criteria  in  Priority  2  and  Priority  3  sections  may  be  determined  by  a  measurement 
facility  to  be  Not  Applicable.  The  scoring  percentages  are  based  on  the  number  of 
applicable  criteria,  not  the  total  number  of  criteria. 
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TABLE  2-2.  EVALUATION  CRITERIA  APPLIED  TO  THE  OPTICAL 
MEASUREMENTS  FACILITY  (OMF)  LAB  BOOK 


Lab 

Book 

Priority 

Section 

No. 

Lab  Book  Section  Title 

1 

9 

Measurement  Traceability  and  Calibration 

10 

Measurement  and  Calibration  Procedures 

1  2.3.1  J 

11 

Handling  of  Calibration  Items 

17 

Interlaboratorv  Comparison  Programs 

18 

Data  Processing  Procedures 

V 

19 

Laboratory-Specific  Calibration  Uncertainty 

2 

r 

5 

Ouality  System,  Audit,  and  Review 

1  2.3.2  ^ 

8 

Equipment  and  Reference  Materials 

13 

Certificates  and  Reports 

20 

Ongoing  Research,  Planned  Improvements 

3 

f 

1 

Introduction  and  Endorsement 

2 

References 

3 

Glossary 

4 

Organization  and  Management  Structure 

1  2.3.3 

6 

Personnel 

7 

Accommodation  and  Environment 

12 

Record  Keeping 

14 

Subcontracting  Records 

15 

Outside  Support  Services  and  Suppliers 

V 

16 

Complaints 

NOTE 


E^, 


The  numbering  system  for  paragraph  2.3.1,  paragraph  2.3.2,  and 
paragraph  2.3.3  follows  the  section  numbering  of  the  OMF  Lab  Book ,  but 
is  presented  here  in  the  priority  order  shown  in  Table  2-2.  Likewise,  the 
formatting  for  these  paragraphs  follows  that  of  the  OMF  Lab  Book. 


2.3.1  Priority  1  Sections 

SECTION  9  -  MEASUREMENT  TRACEABILITY  AND  CALIBRATION  (PRIORITY  1) 

This  section  should  complement  Section  8  as  it  specifically  relates  to  primary  optical 
properties  calibration.  The  laboratory  will  document  its  calibration  program  that  ensures  all 
equipment  critical  to  optical  properties  measurement  calibrations  operates  within  certified 
performance  limits.  Equipment  and  system  calibration  intervals  should  be  clearly  established. 
Whenever  possible,  traceability  of  system  components  and  of  total  system  calibration  to  a 
national  standard  should  be  referenced.  The  system  configuration  used  in  optical  properties 
calibrations  should  be  tracked  using  the  method  documented  in  Section  8.  The  system 
configuration  used  in  inter-laboratory  comparisons  should  be  clearly  documented  together  with 
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the  results  of  calibration  comparisons.  System  components  tracked  under  Lab  Book  Section  8 
need  only  be  referenced  here. 

Note.  Z540  Section  9.5  is  found  in  laboratory  book  ( Lab  Book )  Section  11. 

Evaluation  Criteria: 

9.1:  Does  the  laboratory  have  an  established  and  documented  program  to  ensure  that  its 

calibration  program  supports  the  stated  calibration  performance  of  its  optical  properties 
measurement  system(s)? 

9.2:  Is  the  optical  properties  laboratory  calibration  process  traceable  to  a  published  (MATA, 

IEEE,  etc.)  calibration  artifact  standard  or  other  national  standard  (when  developed)? 


Under  normal  circumstances,  optical  properties  measurement  systems  are 
calibrated  as  a  total  end-to-end  system.  In  special  test  cases  where  specific 
end-to-end  calibration  measurements  are  not  the  norm,  Sections  9.3,  9.4, 
and  9.6  must  be  addressed  for  these  cases. 


9.3:  Does  the  laboratory  measurement  and  calibration  program  include  a  top-level  schedule 

for  calibration  intervals  for  each  piece  of  test  equipment  used  to  assess  system  performance? 

9.4:  Is  there  documentary  evidence  the  top-level  schedule  (9.3)  is  being  rigorously  followed? 

9.6:  Does  the  laboratory  have  a  documented  program  to  monitor  test  equipment  and/or 

calibration  artifacts  between  scheduled  calibrations  and  verifications? 

9.7:  When  the  laboratory  participates  in  inter-laboratory  comparisons  (see  5.6),  is  the 

laboratory  optical  properties  system  configuration  employed  in  those  comparisons  clearly 
documented? 


On-site  Review:  For  the  primary  calibration  artifacts,  are  sample  acceptable  calibrations  or 

verification  records  available  within  the  laboratory  book? 

SECTION  10  -  MEASUREMENT  AND  CALIBRATION  PROCEDURES  (PRIORITY  1) 

Both  measurement  and  calibration  procedures  should  be  fully  documented.  Separate 
documentation  should  be  available  for  the  measurement  and  calibration  of  directional- 
hemispherical  reflectance,  hemispherical-directional  reflectance,  and  the  bidirectional  reflectance 
distribution  function.  Procedures  for  deriving  or  assigning  an  optical  property  value  to  a  primary 
target  using  “transfer”  calibration  standards  should  be  documented  in  this  section.  Calibration 
and  measurement  results  to  support  the  validity  of  procedures  used  should  be  displayed  or 
referenced.  The  validity  of  such  results  should  be  supported  by  stated  uncertainty  bounds 
obtained  through  well-defined  uncertainty  procedures  appropriately  referenced  here  or  in  Section 
19.  Calibration  intervals  should  be  clearly  established. 

Note:  The  numbering/lettering  sequence  below  follows  Z540. 
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Evaluation  Criteria: 

10.1:  Does  the  laboratory  have  documented  system- wide  calibration  procedures? 

10.2a:  Are  separate  calibration  and  measurement  procedures  for  each  major  measurement 
system  or  for  each  type  of  measurement  conducted  available? 

10.2b:  Are  the  calibration  measurement  procedures  consistent  with  the  accuracy  required  for 
each  type  of  calibration  measurement  (as  defined  in  the  uncertainty  analysis  in  Section  19)? 

10.3:  Is  there  documentation  on  the  procedures  for  deriving  or  assigning  optical  property 
values  for  a  measured  target  by  using  transfer  standards? 

10.4:  Are  system- wide  calibrations  scheduled  on  an  appropriate  cycle  for  the  measurement 
system?  Is  this  cycle  defined?  Is  there  evidence  they  are  being  performed  when  scheduled?  Are 
the  results  of  system- wide  calibrations  included  or  referenced  in  this  section?  Are  these 
calibrations  provided  to  the  customer  as  part  of  their  normal  data  report? 

Calibration  schedule  is  not  discussed  in  this  section  although  it  may  be  described  to  some  extent 
in  section  5. 

10.5:  If  sampling  is  included  as  a  part  of  calibration  or  measurements,  is  documentation 
available  that  describes  the  procedures  and  applied  statistical  techniques  employed?  Detailed 
data  processing  implementations  may  be  provided/discussed  in  Section  18,  if  applicable. 

10.6:  Do  the  results  of  acceptable  system-wide  calibrations  support  the  uncertainty  bounds 
established  in  Section  19? 


NOTE 


Computers  are  currently  an  integral  part  of  every  optical  properties 
laboratory.  Since  computers  are  used  to  record,  store,  retrieve,  and  process 
information,  certain  processes,  procedures,  and  documentation  need  to  be 
made  available  to  the  operator.  Normal  user  manuals  are  crucial,  and  on¬ 
line  “help”  features  of  commercial  off-the-shelf  (COTS)  software  will 
fulfill  a  large  portion  of  the  requirements  to  follow.  Specialized  software 
written  exclusively  for  laboratory  data  acquisition  and  processing  requires 
equally  thorough  documentation. 


10.7a:  For  (any)  computer  software  used  to  acquire  and/or  store  the  raw  optical  data,  is 

there  an  available  user  manual(s),  either  as  automated  help  files  or  as  a  separate  document(s), 
which  describes  the  correct  operation  and  use  of  the  software?  If  the  manual  is  not  embedded  in 
the  laboratory  book,  is  the  physical  location  of  the  documentation  clearly  identified? 

10.7bl:  For  computer  software  used  to  store  (locally  or  on  a  network)  or  transmit  from 

one  network  location  to  another,  are  appropriate  users  manuals  and/or  COTS  help  files  available? 

10.7b2:  Is  there  a  database  or  other  appropriate  tracking  list  of  the  computer  program(s) 

that  are  used  in  the  end-to-end  data  acquisition,  transmission,  and  processing  steps? 

10.7b3:  For  computer  software  used  to  process  the  raw  data  files  into  final  data  products, 

are  appropriate  user  manuals  and/or  COTS  help  files  available?  (This  requirement  may  be 
satisfied  one  of  two  ways.  If  specifically  written  users  manuals  have  detailed  specific  examples, 
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these  can  be  used.  If  such  specific  examples  do  not  exist,  the  requirement  can  be  made  by 
appending  sample  “step-by-step”  processes  that,  if  followed,  produce  the  final  data  products.) 

10.7cl:Is  there  an  assigned  computer  support  and/or  network  administrator  available  to  maintain 
computer  equipment  used  in  the  laboratory  data  acquisition  and  processing? 

Cross-reference  with  Section  6,  Personnel.  Is  there  an  IT  person  on  staff,  or  is  there  a  support 
organization  that  provides  IT  support? 

10.7c2:Is  there  a  diagram  of  the  local  area  network  within  the  laboratory,  if  applicable? 

10.7d:  Are  critical  environmental  and/or  operating  procedures  monitored  to  maintain  the  stated 
calibration  accuracy  and/or  stay  within  the  published  calibration  uncertainty  of  Section  19? 

10.7e:  Are  there  procedures  that  ensure  security  of  the  data  to  prevent  the  unauthorized  or 
inadvertent  access,  manipulation,  or  destruction  of  computer  data  or  records? 


Note:  An  approved  network  automated  information  system  (AIS)  or  standard  practices  and 
procedures  (SPP)  plan  will  generally  satisfy  this  requirement.  If  the  AIS/SPP  is  not 
electronically  available  in  the  laboratory  book,  its  physical  location  must  be  precisely  identified. 

On-site  Review:  Request  the  laboratory  manager  perform  a  system-wide  calibration. 

Verify  that  the  documented  procedures  are  followed  by  laboratory  technicians. 

On-site  Review:  Compare  the  results  of  the  system-wide  calibration  with  the  historical 

records  found  in  Sections  12,  13,  and/or  19.  Are  the  results  within  the  stated  calibration 
uncertainty  limits? 

On-site  Review:  Ask  a  technician  if  the  observed  documented  procedures  are  up  to  date. 

Ask  when  the  procedures  were  last  reviewed  for  completeness  and  accuracy. 

SECTION  11  -  HANDLING  OF  CALIBRATION  ITEMS  (PRIORITY  1) 

This  section  should  summarize  how  primary  calibration  items  are  protected  during  storage, 
handling,  and  use.  Mechanical  tolerance  certificates  or  techniques  for  verifying  tolerances  of 
primary  standards  are  appropriately  placed  in  this  section. 

Evaluation  Criteria: 

11.1:  Does  the  laboratory  have  a  system  for  uniquely  identifying  primary  calibration  artifacts? 

1 1.2:  Does  the  laboratory  have  a  system  for  assessing  the  mechanical  tolerances  and  electrical 
conductivity  of  the  primary  calibration  artifacts  upon  initial  receipt?  Are  visual  inspections 
performed  on  the  calibration  artifacts  as  part  of  the  calibration  process?  Is  there  a  stated 
procedure  for  reacting  to  artifacts  found,  suspected  to  be  damaged,  or  suspected  to  be  out  of 
tolerance?  (A  mechanical  test  and/or  measurement  certificate  will  satisfy  the  initial  receipt 
requirement.) 

1 1.3:  Does  the  laboratory  have  documented  procedures  and  appropriate  facilities  to  avoid 
deterioration  or  damage  to  calibration  artifacts  during  storage,  handling,  preparation,  and 
calibration? 

Note:  1 1.3  also  satisfies  Z540  Section  9.5. 
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1 1.4:  Does  the  laboratory  book  identify  the  exact  location  of  the  calibration  artifact  standard 
when  not  in  use? 


The  biggest  threat  posed  to  a  standard  optical  properties  reference  material 
(Spectralon  reference,  front- surfaced  mirror,  etc.)  is  physical  damage. 

Most  calibration  reference  materials  can  be  contaminated  without  any 
visual  indication.  Therefore,  it  makes  sense  to  store  the  calibration 
artifacts  in  well-identified,  clean  storage  devices. 


1 1.5:  Does  the  laboratory  take  reasonable  and  appropriate  measures  to  protect  the  calibration 
artifact(s)  when  not  in  use?  Are  there  documented  procedures  for  reporting  suspected  damage  to 
a  calibration  artifact(s)?  If  the  artifacts  are  repaired  or  replaced,  are  the  procedures  of  1 1.2 
followed  for  the  (repaired  or  replaced)  unit? 

1 1.6:  If  the  primary  calibration  artifacts  employed  by  the  laboratory  are  used  in  any  capacity 
other  than  to  ensure  the  performance  of  the  optical  properties  measurement  system(s),  do  such 
uses  invalidate  or  degrade  the  calibration  artifacts? 


On-site  Review:  Perform  a  visual  inspection  of  the  laboratory  calibration  artifacts  to 

determine  whether  the  conditions,  procedures,  and  facilities  assessed  in  11.1  through  1 1.5  are  as 
stated  in  the  laboratory  book. 

SECTION  17  -  INTERLABORATORY  COMPARISON  PROGRAMS  (PRIORITY  1) 

This  section  shall  include  the  organization’s  policy  regarding  inter-laboratory  comparisons. 
Records  of  participation  in  calibration  inter-laboratory  comparisons,  the  results  obtained,  and 
conclusions  drawn  from  such  studies  are  to  be  presented  or  referenced  here.  If  the  information  is 
already  included  in  Section  10,  a  reference  to  Section  10  is  adequate.  It  is  expected  that  inter¬ 
laboratory  comparisons  will  be  used  regularly  to  improve  measurement  and  calibration  quality  at 
the  laboratory.  An  inter-laboratory  comparison  schedule  should  be  provided. 

Note:  Participation  in  a  national  inter-laboratory  comparison  study  is  required  for  accreditation. 

Evaluation  Criteria: 

17.1:  Does  the  laboratory  have  a  documented  policy  on  inter-laboratory  comparisons? 

17.2:  Does  the  laboratory  participate  in  an  inter-laboratory  calibration  comparison  program  at 
the  national  level? 

17.3:  Are  the  results  of  the  inter-laboratory  calibration  comparisons  of  Section  17.2  available 
for  customer  review? 

17.4:  Are  the  results  of  the  inter-laboratory  comparisons  used  to  evaluate  and  improve 
laboratory  operations? 

On-site  Review:  Review  the  laboratory  documented  inter-laboratory  comparison  policy.  Is 

the  available  data  current?  Does  the  laboratory  have  a  schedule  for  participation  in  inter¬ 
laboratory  comparisons? 
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On-site  Review:  Ask  the  laboratory  technical  manager  how  the  results  of  the  inter¬ 

laboratory  comparisons  are  applied  to  improving  the  quality  of  their  measurements. 

SECTION  18  -  DATA  PROCESSING  PROCEDURES  (PRIORITY  1) 

Important  details  of  processing  measurement  and  calibration  data  should  be  documented  here. 
Sample  results  of  processing  should  be  included.  Standard  data  processing  procedures  should  be 
referenced,  and  the  validity  of  innovative  processing  should  be  documented.  To  evaluate  the 
validity  of  its  data  processing  procedures,  the  laboratory  should  use  a  standard  data  set 
acceptable  to  the  optical  properties  industry.  This  standard  data  set  should  be  used  by  the 
laboratory  to  perform  inter-laboratory  comparison  of  its  processing  algorithms  with  those  used 
by  other  laboratories.  This  section  should  strongly  support  and  complement  the  material  in 
Section  17  above. 

Evaluation  Criteria: 

18.1:  Does  the  laboratory  have  all  primary  optical  properties  data  processing  procedures 
documented? 

18.2:  Have  the  data  processing  modules  been  validated  with  a  standard  data  set  to  ensure  the 
correctness  of  the  algorithms  employed? 

18.3:  Are  the  results  of  data  processing  compared  against  known  quantities  or  standards  to 
verify  accuracy  of  the  measurement  and  data  processing  process? 

On-site  Review:  Is  there  a  procedure  to  collect,  determine,  and  validate  all  customer  data 

processing  requirements,  to  include  such  key  items  as  specific  formats,  parameters,  and 
quantities?  Is  there  a  procedure  for  translating  user  requirements  into  the  specific  procedures  to 
collect  the  data  identified  by  the  customer? 

On-site  Review:  Are  procedures  in  place  for  validating  the  data  in  (near)  real-time  to  ensure 

the  proper  data  is  being  collected? 

On-site  Review:  Ask  a  laboratory  technician  to  process  a  file  from  a  standard  data  set  and 

compare  the  results  to  those  known  to  be  correct. 

On-site  Review:  Ask  the  laboratory  technical  manager  how  new  or  improved  data 

processing  algorithms  are  validated. 

On-site  Review:  Review  the  results  of  any  inter-laboratory  data  processing  comparisons. 

Are  these  results  available  to  laboratory  customers? 

SECTION  19  -  LABORATORY-SPECIFIC  CALIBRATION  UNCERTAINTY  ANALYSIS 
(PRIORITY  1) 

Policies  and  procedures  for  establishing  laboratory  uncertainties  need  to  be  completely 
documented  or  referenced  here.  A  sample  uncertainty  table  together  with  system  parameters  or 
other  adequate  uncertainty  methods  should  be  linked  or  available  in  this  section.  One  approach 
to  satisfy  this  requirement  is  the  “Report  of  Measurement”  described  by  B.  M.  Welsh  and  B.  M. 
Kent  in  “An  RCS  Uncertainty  Analysis  and  Calibration  Certificate  for  AFRL  Calibration 
Cylinder,”  Year  2000  Antenna  Measurement  Techniques  Association  (AMTA)  Symposium, 
Philadelphia,  PA.  Any  system  parameters  should  be  displayed  here.  Any  scientifically  based 
uncertainty  analysis  is  acceptable,  providing  all  appropriate  assumptions  and/or  exclusions 
regarding  the  parameters  comprising  the  analysis  are  clearly  documented. 
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Evaluation  Criteria: 

19.1:  Does  the  laboratory  have  a  policy  for  the  development  and  use  of  a  calibration 
uncertainty  analysis?  (Alternate  location  for  Section  5.2u.) 

19.2:  For  the  primary  calibration  standard(s)  used  by  the  laboratory,  is  there  a  nominal  optical 
properties  measurement  with  estimated  uncertainties  in  either  graphical  or  tabular  form  for  that 
standard? 

Note:  A  NIST  calibration  certificate  may  be  used  to  fulfill  this  requirement,  but  other  formats 
containing  similar  information  are  acceptable. 

19.3:  Is  the  calibration  uncertainty  shown  in  19.2  consistent  with  stated  calibration  capabilities 
of  the  laboratory  in  other  laboratory  book  sections? 

Note:  If  the  laboratory  has  a  stated  calibration  accuracy  less  than  the  accuracy  shown  in  this 
nominal  measurement,  those  capabilities  must  be  changed  to  match  those  obtained  in  the  Section 
19.2  measurements. 

19.4:  Are  the  results  of  the  uncertainty  analysis  for  primary  calibration  reflected  in  the  day-to- 
day  measure  of  calibration  quality  discussed  in  Section  10?  In  other  words,  are  typical 
uncertainties  used  as  a  quality  check  for  daily  calibration  measurements? 

19.5:  Does  the  laboratory  have  a  “Report  of  Measurement”  for  its  most  commonly  used 
calibration  artifact  stating  its  expected  uncertainty  under  nominal  operating  conditions? 


On-site  Review:  Ask  the  radar  technicians  to  show  (in  tabular  or  graphical  form)  the 

predicted  uncertainty  of  the  primary  calibration  standard.  Can  the  technicians  find  the 
information? 

On-site  Review:  Do  the  technicians  use  the  primary  calibration  target  uncertainty  bars  to 

bracket  acceptable  (daily)  calibration  as  a  quality  check? 

On-site  Review:  Ask  the  following  question  and  assess  answer:  Are  the  actions  taken  by 

the  technicians  when  calibration  measurements  fall  outside  expectations  consistent  with  the 
stated  laboratory  policy  on  exceptions  to  approved  calibration  process?  (See  Section  5  and 
Section  10.) 

2.3.2  Priority  2  Sections 

SECTION  5  -  QUALITY  SYSTEM,  AUDIT,  AND  REVIEW  (PRIORITY  2) 

The  purpose  of  this  section  is  to  ensure  that  a  complete  overview  of  the  laboratory  quality  system 
and  self-auditing  process  is  provided.  This  section  should  contain  the  following: 

•  Procedures  for  the  control  and  maintenance  of  documentation, 

•  Arrangements  for  ensuring  that  the  laboratory  reviews  all  new  work  to  ensure  that  it 
has  the  appropriate  facilities  and  resources  before  commencing  such  work, 

•  Management  arrangements  for  permitting  departures  from  documented  policies  and 
procedures  or  from  standard  specifications, 

•  Procedures  for  protecting  confidentiality  and  proprietary  rights,  procedures  for  audit 
and  review, 
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•  Procedures  for  a  management  quality  audit  at  least  once  a  year, 

•  Procedures  for  documentation  of  all  audit  processes,  and 

•  Procedures  for  tracking  and  monitoring  the  quality  of  the  primary  optical  properties 
calibration  data. 

The  Optical  Properties  Certification  Review  Committee  Certification  Report  should  be  attached 
here  upon  completion  of  the  certification  review  process. 

Note  1:  All  items  of  Section  5  need  to  be  addressed  at  some  point  in  the  laboratory  book. 
Because  of  their  importance,  some  of  these  items  are  treated  in  separate  sections  of  the 
laboratory  book.  Any  such  information  requested  in  Section  5,  but  appearing  elsewhere  in  the 
laboratory  book  must  be  appropriately  documented.  For  example,  Section  5.2(a)  appears  in  this 
book  in  Section  1.  Here  we  include  all  items  that  are  not  treated  separately,  as  listed  after  the 
section  heading.  The  purpose  of  this  section  is  to  ensure  that  a  complete  overview  of  the 
company’s  quality  system  and  its  self-auditing  process  are  given.  Important  miscellaneous  items 
are  to  be  included  here.  We  recommend  closely  following  the  appropriate  parts  of  Section  5  of 
the  main  text.  Section  5.2(s)  should  be  the  section  where  the  certification  audit  report  is 
attached. 

Note  2: 


Z540  5.2e 

is  found  in  Laboratory  Book 

Section  6 

Z540  Section  5.2f 

is  found  in  Laboratory  Book 

Section  1 

APPENDIX  I  -  Z540  Section  5.2g 

is  found  in  Laboratory  Book 

Section  9 

Z540  Sections  5.2h,  j,  1,  m,  and  n 

are  found  in  Laboratory  Book 

Section  10 

Z540  Section  5.2k 

is  found  in  Laboratory  Book 

Section  11 

Z540  Section  5.2q 

is  found  in  Laboratory  Book 

Section  16 

Z540  Section  5.2r 

is  found  in  Laboratory  Book 

Section  4 

Z540  Section  5.2u 

is  found  in  Laboratory  Book 

Section  19 

Note  3:  The  numbering/lettering  sequence  below  follows  Z540. 

Evaluation  Criteria: 

5.2a:  Does  the  laboratory  book  contain  a  top  management  quality  statement  here  or  in 
Section  1? 

5.2b:  Are  appropriate  organizational  charts  in  place  here  or  in  Section  4? 

5.2c:  Are  the  roles  and  responsibilities  of  management,  technical  operations,  support  services, 
and  quality  systems  identified  here  or  in  Section  4? 

5. 2d:  Does  the  laboratory  book  describe  how  documents  are  controlled  and  referenced? 
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5.2i:  Does  the  laboratory  organization  have  a  process  to  assess  whether  it  has  the  technical 
capability  and  resources  (schedule  time)  to  accept  new  work? 

5.2o:  Is  there  a  clearly  established  policy  and  process  when  measurement  discrepancies  are 
seen  or  departures  from  documented  policies  and  procedures  occur? 

5.2p:  Is  there  a  management  policy  for  permitted  departures  from  documented  policies? 

5.2s:  Are  periodic  review  processes  documented? 

Note:  The  Optical  Properties  Certification  Review  Committee  Certification  Report  should  be 
attached  here  upon  completion  of  the  review  process. 

5.2-t:  Is  the  laboratory’s  policy  for  establishing  and  changing  calibration  intervals  documented 
for  systems  or  equipment  it  controls? 

5.3:  Are  the  laboratory’s  internal  (annual)  and  external  (every  three  year)  audit  processes 

documented?  Have  previous  audit  deficiencies  been  corrected  by  the  next  annual  audit? 

5.4:  Was  a  management  quality  audit  performed  within  the  previous  12  months? 

5.5:  Are  previous  audits  and  their  findings  a  permanent  part  of  the  laboratory  book? 

5.6:  Does  the  laboratory  have  a  process  for  tracking  and  monitoring  the  quality  of  the  primary 

optical  properties  calibration  data  for  calibration  artifacts  used  in  the  normal  measurement 
process? 

Is  there  a  day-to-day  pass/fail  check  criteria  established  for  calibration  measurements? 

Have  the  results  been  verified,  either  through  repeated  measurements,  inter-laboratory 
comparisons  or  both? 

On-site  Review:  Ask  laboratory  technicians  what  is  the  pass/fail  criteria  for  calibration. 

On-site  Review:  Ask  laboratory  technicians  what  is  done  if  calibrations  depart  from 
expectations. 

On-site  Review:  Review  internal  laboratory  process  of  assessing  new  work  (or  a  new  customer 
requirement).  Is  it  complete?  Can  it  easily  be  provided  to  a  customer  on  request? 

SECTION  8  -  EQUIPMENT  AND  REFERENCE  MATERIALS  (PRIORITY  2) 

Configuration  control  is  essential  for  maintaining  a  repeatable,  quality  optical  properties  system. 
Therefore,  this  section  of  the  laboratory  book  may  be  one  of  the  most  significant.  Here  the 
laboratory  should  identify  all  applicable  electro-optical  equipment  that  makes  up  the  optical 
properties  laboratory.  References  to  appropriate  equipment  manuals  should  be  given  in 
sufficient  detail  so  that  a  qualified  technician  can  trace  the  major  subsystems  of  the  laboratory. 
Maintenance  and  calibration  histories  should  be  kept  for  critical  components,  and  complete 
system  configuration  should  be  tracked.  The  procedures  for  identifying  equipment,  maintenance, 
and  configuration  control  should  be  documented  here.  Personnel  responsible  for  following  these 
procedures  should  be  clearly  identified.  For  electronic  (digital/intra-net)  laboratory  books,  it  is 
not  essential  to  include  an  electronic  version  of  critical  laboratory  equipment  in  this  section. 
However,  the  titles  and  exact  locations  of  externally  referenced  documents  must  be  included  in 
this  section. 
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Note:  The  numbering/lettering  sequence  below  follows  Z540. 

Evaluation  Criteria: 

8.1:  Does  the  laboratory  have  a  list,  database,  or  other  log  or  configuration  control  method  for 

identifying  and  tracking  the  main  laboratory  equipment  systems  and  subsystems? 

8.2a:  For  any  laboratory  equipment  identified  in  8.1,  is  a  maintenance  schedule  identified  for 
periodic  or  preventive  maintenance? 

8.2b:  Is  there  any  evidence  the  equipment  maintenance  schedules  are  being  followed  (i.e.,  last 
“check-on”  date,  maintenance  logs,  etc.)? 

8.2c:  For  major  laboratory  systems  or  subsystems,  are  maintenance  procedures  clearly 
identified  in  either  a  reference  manual  or  a  printable  procedure? 


Reference  manuals  may  be  included  in  Section  10;  cross-references  here  would  be  useful. 

8.3a:  Is  any  individual  equipment  in  the  laboratory  system  identified  as  to  whether  its  internal 
calibration  status  affects  the  level  of  laboratory  optical  properties  calibrations?  (This 
requirement  is  N/A  if  end-to-end  system  calibration  is  performed  as  a  routine  practice  at  the 
laboratory.) 

8.3b:  For  any  equipment  identified  in  8.3-a,  is  the  calibration  status  and  interval  of  said 
equipment  marked  or  displayed  on  the  equipment?  (If  8.3a  is  N/A,  this  section  is  N/A.) 

8.4a:  Is  there  an  appropriate  reference  manual  for  each  piece  of  laboratory  equipment  identified 
in  8.1?  This  requirement  may  be  satisfied  with  a  database  or  listing  cross-referencing  the 
equipment  with  its  manual  and  its  exact  physical  location  when  not  in  use. 

8.4b:  Does  the  document  and  equipment  log  show  the  name  and  current  location  of  the 
equipment? 

8.4c:  Does  the  document(s)  show  the  maintenance  history  of  these  pieces  of  equipment? 

8.4d:  Does  the  document  show  a  history  of  damage,  malfunction,  modification,  or  repair  of  the 
items? 

8.4e:  If  automated  testing  is  used  to  verify  subsystem  performance  (e.g.,  loop  test  modes, 
automated  power  supply  monitoring,  etc.),  are  non-compliant  loop  tests  documented  and 
corrected  (or  noted)  prior  to  system  use? 

On-site  Review:  Select,  at  random,  five  pieces  of  equipment  from  the  log  of  8.1  above. 

Can  the  pieces  of  equipment  be  located? 

On-site  Review:  Ask  to  see  the  maintenance  procedures  (if  applicable)  for  those  pieces  of 

equipment.  Are  the  manuals  current? 

On-site  Review:  Ask  to  see  the  reference  manuals  for  the  five  pieces  of  equipment  selected. 

Are  the  manuals  readily  available?  Were  those  manuals  found  in  the  locations  stipulated  in  the 
laboratory  book? 

On-site  Review:  For  the  five  pieces  of  equipment  selected,  is  there  a  historical  log  kept  on 

damage,  repair,  or  recalibration  (if  appropriate)? 
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SECTION  13  -  CERTIFICATES  AND  REPORTS  (PRIORITY  2) 

This  section  should  define  the  standard  reporting  formats  used  by  the  laboratories  when  reporting 
optical  properties  measurements  to  customers.  While  any  report  can  be  specialized  to  the 
requirements  of  the  customer,  this  section  should  provide  the  baseline  information  that  will 
always  be  available  in  a  test  report.  A  laboratory  is  allowed  to  use  its  own  format  if  the 
minimum  information  required  by  ANSI  Z540-1  is  included. 

Evaluation  Criteria: 

13.1:  Does  the  laboratory  book  include  a  description  of  and  purpose  for  each  type  of  certificate 
or  report  employed  or  delivered  by  the  laboratory  (e.g.,  measurement  data  and  test  summary 
reports)? 

13.2a:  Does  each  certificate  or  report  include: 

•  A  title  (e.g..  Measurement  Report)? 

•  Name  and  address  of  the  facility? 

•  Points  of  contacts? 

•  Phone/fax  numbers? 

•  Location(s)  of  the  test(s)? 

•  Name,  address,  and  points  of  contact  of  the  customer? 

•  Description  and  unambiguous  identification  of  the  measured  item  (including  its 
configuration,  as  appropriate)? 

•  Characterization  and  condition  of  the  measured  item? 

•  Date(s)  and  times(s)  of  the  measurement(s)? 

•  Identification  of  the  calibration  procedure(s)  used  or  unambiguous  description  of  non¬ 
standard  procedures  employed;  reference  to  sampling  and/or  data  processing  procedures, 
where  relevant? 

•  Any  deviations  from,  additions  to,  or  exclusions  from  documented  operating  procedures? 

•  Other  information  relevant  to  the  measurement(s)  such  as  environmental  conditions  or 
failures/anomalies  encountered  during  the  measurement(s)? 

13.2b:  Does  each  certificate  or  report  include  measurements,  examinations,  and  derived  results 
supported  by  tables,  graphs,  sketches,  and  photographs  (and/or  digital  representations  of  such), 
as  appropriate  for  the  measurements  conducted? 

13.2c:  Does  each  certificate  or  report  include  a  statement  of  the  estimated  uncertainty  of  the 
primary  calibration  results  and  the  traceability  of  the  measurements? 

13. 2d:  Does  each  certificate  or  report  include  a  signature  and  title  of  the  person(s)  accepting 
responsibility  for  the  content  of  the  certificate  or  report,  and  the  date  of  issuance? 

13. 2e:  Does  each  certificate  or  report  include  a  statement  regarding  any  limitations  on  the  use  or 
interpretation  of  the  measurement  data? 

13.3:  Are  portions  of  the  measurement  data  products  performed  by  subcontractors?  If  so,  are 
such  portions  clearly  identified?  (This  will  probably  be  N/A  for  most  laboratories.) 

13.4:  Is  there  a  procedure  for  amending  a  report?  Is  there  a  procedure  for  notifying  customers 
promptly,  in  writing,  of  any  event  such  as  discovery  of  defective  calibration  equipment  that  may 
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cast  doubt  on  the  validity  of  the  results  provided  in  any  given  report  or  certificate,  including  the 
magnitude  of  the  errors  that  may  exist? 

13.6:  Is  there  a  documented  procedure  for  transmitting  reports  or  certificates  to  customers  or 
third  party  recipients  such  that  confidentiality  is  preserved? 

Note:  A  hand  receipt,  computer  receipt,  registered  mail  receipt,  or  other  appropriate  tracking 
system  may  be  used  to  fulfill  this  requirement. 

On-site  Review:  No  on-site  review  criteria  for  this  section. 

SECTION  20  -  ONGOING  RESEARCH,  PLANNED  IMPROVEMENTS  (PRIORITY  2) 

The  purpose  of  this  section  is  to  briefly  summarize  the  three-year  look  ahead  regarding  any 
laboratory  research  and/or  other  activities  designed  to  improve  laboratory  data  quality, 
efficiency,  repeatability,  or  traceability.  A  list  of  desirable  research  areas  to  be  conducted  in  the 
future  to  improve  specific  and  known  deficiencies  on  the  laboratory  or  in  the  optical  properties 
industry  should  also  be  provided,  as  well  as  a  single  top-level  roadmap.  Ongoing  plans  for  future 
upgrades  of  equipment  should  be  included.  Such  research  information  will  allow  customers  to 
quickly  identify  and  evaluate  ongoing  laboratory  improvements  in  the  context  of  their  current  or 
planned  use  of  the  laboratory.  It  also  may  offer  the  customer  an  opportunity  to  cost  share  or 
jointly  sponsor  laboratory  research  of  interest  to  the  customer. 

Evaluation  Criteria: 

20-1:  Is  there  a  description  or  summary  of  future  planning  such  as  three-year  plans,  upgrades, 
and  quality  or  laboratory  improvements? 

20-2:  Is  there  a  roadmap  formatted  to  summarize  the  three-year  plan? 

Note:  The  roadmaps  may  be  longer  than  three  years  but  must,  at  a  minimum,  look  ahead  three 
years. 

On-site  Review:  As  a  result  of  the  review  committee  feedback,  is  there  a  summary  of  topics  and 
issues  that  will  be  worked  on  a  time-available  basis  over  the  succeeding  three  years? 

2.3.3  Priority  3  Sections 

SECTION  1  -  INTRODUCTION  AND  ENDORSEMENT  (PRIORITY  3) 

This  section  should  contain  a  brief  statement  of  compliance  with  the  standards  set  by  this 
document  in  accordance  with  ANSI/NCSL  Z-540- 1-1994,  as  well  as  a  clear  policy  statement 
regarding  the  organization's  commitment  to  continuous  quality  improvement.  The  laboratory 
book  should  be  signed  as  “approved”  by  an  appropriate  manager  or  director  at  or  near  the  top  of 
the  reporting  chain  of  the  laboratory  quality  manager  or  technical  lead  engineer.  The  exact 
format  and  wording  may  be  customized  to  suit  the  organization. 

Evaluation  Criteria: 

1.1:  Does  the  laboratory  book  identify  the  host  organization? 
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1.2:  Is  the  endorsement  signed  by  an  appropriate  manager  or  director  who  has  the 

responsibility  and  authority  to  direct  changes  and  corrections  if  laboratory  or  product 
deficiencies  are  found?  (Alternate  location  for  Section  5.2f.) 

1.3:  Is  there  a  clear  commitment  to  quality  in  the  introduction  and  endorsement? 

On-site  Review:  Ask  to  meet  the  endorser.  Is  the  endorser  clearly  committed  to  the  quality 

process? 

SECTION  2  -  REFERENCES  (PRIORITY  3) 

This  section  should  list  the  documents  cited  in  the  laboratory  book.  In  particular,  it  may  be 
convenient  to  include  stand-alone  reports  covering  various  aspects  of  laboratory  operations 
(operational  security,  measurement  procedures,  uncertainty  analysis  procedures,  equipment 
documentation,  etc.),  which  could  be  included  in  the  laboratory  book  by  reference  only. 

Evaluation  Criteria: 

2.1:  Does  the  laboratory  book  include  a  list  of  documents  and  references  external  to  the 

laboratory  book? 

2.2:  Are  all  references  unambiguous  enough  to  identify  the  documents? 

2.3:  Are  the  documents  listed  cross-referenced  or  linked  to  the  appropriate  laboratory  book 

section? 

References  maybe  hyper-linked  to  the  actual  document,  or  if  the  document  is  not  digital  or 
otherwise  not  available  via  electronic/digital  connection,  then  the  location  must  be  identified. 

On-site  Review:  Ask  to  see  five  documents  at  random  on  the  list. 


SECTION  3  -  GLOSSARY  (PRIORITY  3) 

This  section  should  list  and  define  the  specialized  terms  and  acronyms  used  in  the  laboratory 
book.  All  acronyms  should  be  defined  upon  first  use.  Specialized  terms  related  to  the 
acquisition  and  processing  of  optical  properties  data  should  also  be  defined.  The  information  in 
this  section  should  aid  the  reviewers  in  translating  technical  jargon  from  the  body  of  the  report. 

Evaluation  Criteria: 

3.1:  Does  the  laboratory  book  contain  an  appropriate  glossary? 

3.2:  Do  acronyms  (jargon)  occur  in  any  laboratory  book  section  that  are  not  defined 

(immediately)  in  that  section  or  listed  here  in  the  glossary? 

On-site  Review:  No  on-site  review  criteria  for  this  section. 

Note:  Reviewers:  Please  list  any  undefined  acronyms  below. 


Note:  Laboratory  Book  Authors:  After  review,  please  include  in  the  glossary  any  acronyms  or 
terms  identified  by  reviewers  as  undefined. 
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SECTION  4  -  ORGANIZATION  AND  MANAGEMENT  STRUCTURE  (PRIORITY  3) 

This  section  should  fully  define  the  organization  that  is  chartered  to  operate  the  laboratory.  The 
management  hierarchy  from  the  laboratory  line  technicians  and  engineers  up  through  the  highest 
levels  of  laboratory  management  should  be  defined.  Lead  personnel  and  approved  signatories 
should  be  identified.  The  laboratory  quality  manager  and  technical  managers  must  be  identified. 
Ideally,  work  phone  numbers  should  be  available  for  all  critical  laboratory  personnel.  Standard 
company  organization  charts  are  acceptable.  In  addition,  a  policy  statement  on  customer 
confidentiality  and  or  proprietary  rights  protection  must  be  included  here. 

Evaluation  Criteria: 

4. 1 :  Does  the  laboratory  book  contain  a  complete  reporting  chain  or  organizational  chart  of 
the  laboratory? 

4.2:  Does  the  laboratory  identify  a  laboratory  technical  manager? 

4.3:  Does  the  laboratory  identify  the  laboratory  quality  manager? 

4.4:  Does  the  laboratory  identify  the  management  chain  from  the  technician  level  to  the 

highest  level  of  the  organization? 

4.5:  Does  the  laboratory  quality  manager  have  direct  and  unfettered  access  to  the  individual 
signing  the  endorsement  in  Section  1,  Introduction  and  Endorsement? 

4.6:  Is  there  a  clear  policy  on  non-disclosure  and/or  company  proprietary  data  or  information? 

4.7:  If  laboratory  operators,  engineers,  and/or  technicians  are  from  a  different  organization  or 

company  than  the  laboratory  owners,  are  the  respective  roles  and  responsibilities  of  the  two 
organizations  and  their  interfaces  clearly  spelled  out? 

4.8:  Are  personnel  qualifications  available  or  referenced  in  other  laboratory  book  sections? 

4.9:  In  cases  of  absence,  are  alternate  technical  and  quality  managers  identified? 

On-site  Review:  Ask  laboratory  technicians  to  identify  the  technical  and  quality  managers. 

On-site  Review:  Ask  the  quality  manager  if  he/she  has  direct  access  to  the  endorser  of 

Section  1. 

On-site  Review:  Check  to  see  whether  the  laboratory  technicians  and  engineers  know  who 

the  primary  and  alternate  technical  and  quality  managers  are. 

On-site  Review:  Is  the  laboratory  supervision  done  by  person(s)  familiar  with  the 

calibration  methods  and  procedures,  the  objective  of  the  calibration,  and  how  to  assess  the 
results? 

SECTION  6  -  PERSONNEL  (PRIORITY  3) 

This  section  should  contain  the  qualifications  of  all  personnel  assigned  to  the  laboratory. 
Specialized  training,  training  certificates,  or  other  laboratory -related  qualifications  should  appear 
in  this  section.  Company  or  resume  format  may  be  used.  If  an  organization,  due  to  personnel 
policy  or  union  concerns,  treats  their  records  as  “company  sensitive”  or  “company  confidential,” 
the  laboratory  book  should  so  state,  and  indicate  what  individual  and  organization  has  these 
records,  and  who  has  the  right  to  review  and  update  them.  It  is  the  responsibility  of  the 
laboratory  to  certify  that  their  staff  has  the  qualifications  to  do  their  jobs.  Updated  training 
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records  are  one  indicator  of  the  currency  of  training.  If  the  reviewers  are  not  allowed  to  see  these 
records,  the  laboratory  technical  manager  may  certify,  in  a  signed  statement  herein,  that  such 
records  exist  in  a  remote  location,  and  that  these  records  demonstrate  that  the  employees  meet 
the  minimum  qualification  standards  for  their  respective  jobs.  Note  that  any  training  program 
should  not  only  address  day-to-day  competencies,  but  also  allow  for  technical  growth  to  build 
expertise  to  support  future  tasks,  like  those  identified  in  Section  20. 

Evaluation  Criteria: 

6.1:  Does  the  laboratory  employ  an  appropriate  mix  of  technician,  engineering,  and 

management  personnel  with  appropriate  experience  to  perform  their  assigned  functions? 
(Alternate  location  for  5.2-e) 

Note  of  caution:  Are  the  personnel  qualifications  written  to  match  the  available  workforce,  or 
do  they  truly  represent  the  skill  sets  needed  to  perform  the  work  in  a  competent  manner? 

6.2:  Does  the  laboratory  have  a  formal  training  program,  and  if  so,  are  records  kept  up  to 

date?  Is  the  training  offered  sufficient  to  ensure  or  maintain  technical  competence? 

6.3:  Are  the  training  records  available  to  laboratory  management? 

Note:  These  records  do  not  have  to  reside  in  the  laboratory  book  but  must  be  identified  by 
location. 

On-site  Review:  Ask  to  locate  the  personnel  training  records  of  6.3. 


SECTION  7  -  ACCOMMODATION  AND  ENVIRONMENT  (PRIORITY  3) 

This  section  should  document  the  laboratory  environment  as  well  as  any  and  all  parameters  of 
the  laboratory  environment  that  may  affect  the  outcome  of  measurements  and  thereby  should  be 
monitored.  Some  environmental  variables  that  could  affect  optical  measurements  include 
electromagnetic  interference,  temperature  variations,  and  mechanical  vibrations,  etc. 

(Depending  on  the  complexity  of  the  laboratory,  this  section  may  or  may  not  be  significant.) 

Evaluation  Criteria: 

7.1:  (Deleted)  Same  criteria  is  found  in  Section  10-7d. 

7.2:  Are  the  nominal  measurement  environmental  factors  specified  for  acceptable  operations 

(e.g.,  allowable  temperature  variations,  etc.)? 

7.3:  Are  appropriate  environmental  parameters  recorded  as  a  part  of  the  test  process? 

On-site  Review:  Are  the  environmental  factors  identified  being  recorded  in  a  log  or  with 

the  data?  For  environmental  measurements  being  recorded,  review  such  data  and  operations 
with  laboratory  technicians. 

On-site  Review:  Is  there  any  measurement  data  showing  system  stability  or  repeatability 

versus  stated  control  values  for  environmental  variations?  (In  other  words,  are  the  factors  in  7.2 
derived  based  on  data  and  if  so,  what  data?) 


SECTION  12  -  RECORD  KEEPING  (PRIORITY  3) 
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This  section  describes  the  laboratory’s  procedures  for  creating  and  maintaining  records  of  all 
aspects  of  laboratory  operations.  Procedures  for  maintaining  calibration  records,  measurement 
records,  system  configuration,  etc.,  are  especially  important. 

Evaluation  Criteria: 

12.1:  Does  the  laboratory  have  well-documented  procedures  for  creating  and  maintaining 
records  of  laboratory  operations,  to  include  records  of  calibration,  measurements,  and  system 
configuration? 

12.2:  Are  these  records  indexed  or  stored  so  they  are  easily  located  and  accessible  to  those  with 
a  need  to  review  them? 

On-site  Review:  No  on-site  review  criteria  for  this  section. 

Note:  This  section  may  be  evaluated  in  conjunction  with  the  examination  of  the  records  required 
by  other  sections,  such  as  13,  14,  15,  16,  17,  etc. 


SECTION  14  -  SUBCONTRACTING  RECORDS  (PRIORITY  3) 

This  section  details  any  aspects  of  the  optical  properties  calibration  process  that  are 
subcontracted.  Any  subcontracted  work  must  satisfy  the  ANSI  Z540-1  standard.  (This  section 
may  not  be  applicable  for  most  laboratories.)  The  main  intent  of  this  section  is  to  address 
calibration  activities  sent  to  locations  or  personnel  that  are  not  within  the  laboratory  facility  (on¬ 
site).  When  on-site  contractors  share  complete  responsibility  for  operating  a  laboratory  facility, 
as  is  often  the  case,  we  refer  to  this  situation  as  an  integrated  product  team.  For  example,  AFRL 
hires  an  on-site  contractor  to  assist  in  laboratory  operations.  This  is  not  considered 
subcontracting.  However,  if  a  primary  laboratory  relies  on  an  off-site  secondary  optical 
properties  laboratory  for  calibration  data  or  other  measurements  applicable  to  the  primary 
laboratory,  this  situation  would  be  treated  as  a  subcontracted  measurement,  and  Section  14 
would  apply.  For  most  laboratories,  these  criteria  probably  will  not  apply. 

Evaluation  Criteria: 

14.1:  If  the  laboratory  employs  any  subcontracting  in  the  performance  of  optical  properties 
measurements,  is  there  a  documented  procedure  or  policy  that  stipulates  and  verifies  that  such. 
Does  the  laboratory  maintain  records  of  the  work  accomplished  by  subcontractors? 

14.2:  If  the  laboratory  employs  a  contractor  to  perform  measurements,  does  the  contract  or 
other  operational  document  bind  the  contractor  to  the  procedures  defined  by  the  laboratory  book? 

On-site  Review:  Ask  the  laboratory  manager  whether  subcontractors/contractors  support  or 
perform  optical  properties  measurements.  If  applicable,  review  the  contract(s)  to  verify 
compliance  with  the  ANSI-Z540  requirements. 

SECTION  15  -  OUTSIDE  SUPPORT  SERVICES  AND  SUPPLIERS  (PRIORITY  3) 

This  section  should  document  any  services  or  suppliers  used  to  produce  calibrated  optical 
properties  data.  This  section  complements  Section  14.  For  instance,  if  a  laboratory  purchases  a 
reflectance  standard  from  a  vendor  for  primary  calibration,  this  section  would  document  the 
needed  tolerances  of  such  a  standard,  as  well  as  the  delivered  tolerance  of  the  calibration  item. 
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Evaluation  Criteria: 

15.1:  Are  there  documented  policies  that  ensure  the  quality  of  the  calibrations  and 
measurement  products  if  the  laboratory  uses  outside  services  or  suppliers  that  might  adversely 
affect  the  quality  and  accuracy  of  optical  properties  measurements. 

15.2:  Does  the  laboratory  book  identify  outside  services  or  suppliers,  or  is  there  a  definitive 
statement  that  no  outside  suppliers  or  services  are  employed  in  the  calibration  and  measurement 
functions  of  the  laboratory? 

15.3:  Where  applicable,  are  there  procedures  for  maintaining  records  associated  with  outside 
services  or  suppliers? 


On-site  Review:  If  outside  suppliers  or  services  are  used  at  the  laboratory  under  review,  ask 

to  see  the  associated  records. 


SECTION  16  -  COMPLAINTS  (PRIORITY  3) 

This  section  should  document  the  formal  complaint  procedure  used  to  resolve  disputes  between 
the  laboratory  and  a  customer.  The  complaint  record  keeping  system  also  should  be  defined  or 
referenced  here. 

Evaluation  Criteria: 

16.1:  Does  the  laboratory  have  documented  procedures  for  the  resolution  of  customer 
complaints? 

16.2:  Are  records  maintained  of  all  complaints  and  of  the  actions  taken  by  the  laboratory  to 
resolve  the  complaints? 

16.3:  If  a  customer  complaint  raises  a  concern  regarding  the  laboratory’s  compliance  with 
laboratory  policies  or  procedures,  or  with  the  requirements  of  this  standard,  does  the  laboratory 
have  procedures  to  ensure  that  these  complaints  are  reviewed  and  acted  upon  by  a  level  of 
authority  at  least  as  high  as  the  signatory  identified  in  Section  1?  Is  this  individual  able  to  effect 
the  required  changes  in  laboratory  policy  and  procedures,  if  needed? 

16.4:  If  the  customer  complaint  raises  a  concern  regarding  the  quality  of  the  laboratory’s 
calibrations,  does  the  laboratory  ensure  that  these  complaints  are  reviewed  and  acted  upon  by  a 
level  of  authority  at  least  as  high  as  the  signatory  identified  in  Section  1?  Is  this  individual  able 
to  effect  the  required  changes  in  laboratory  policy  and  procedures,  if  needed? 


On-site  Review:  Examine  records  of  a  random  complaint  (if  applicable)  to  ensure  that  the 

complaints  were  reviewed  and  that  appropriate  action  was  taken  and  recorded. 

On-site  Review:  Ask  laboratory  personnel  if  they  are  familiar  with  the  laboratory’s 

complaint  resolution  process. 

On-site  Review:  Ask  the  laboratory  quality  manager  if  and  how  he/she  is  involved  in  the 

complaint  resolution  process.  Does  the  laboratory  quality  manager  review  all  customer 
complaints  for  indications  that  current  policies  and  procedures  are  in  need  of  review? 


2-19 


RCC  Special  Report:  Development  of  Electro-optical  (EO)  Standard  Processes  for  Application,  November  201 1 


This  page  intentionally  left  blank. 


2-20 


RCC  Special  Report:  Development  of  Electro-optical  (EO)  Standard  Processes  for  Application,  November  201 1 


CHAPTER  3 

MODIFIED  ANSI  Z-540.1-1994-BASED  EVALUATION  CRITERIA  FOR  USE  BY  EO/IR 

FACILITIES 

3.1  Criteria  Modifications  Based  on  Lessons  Learned 

The  following  evaluation  criteria  were  derived  from  the  ANSI  Z-540.1-1994-based 
evaluation  criteria  presented  in  Chapter  2,  modified  by  the  evaluators  subsequent  to  their 
application  of  the  criteria  in  Chapter  2.  The  criteria  presented  here  were  modified  based  on 
lessons  learned  during  the  three  evaluations  of  the  OMF  Lab  Book,  with  the  modifications 
seeking  to  reduce  ambiguity,  delete  redundancies,  and  improve  clarity. 

The  recommendation  of  the  participants  is  that  evaluation  criteria  based  on  either 
ANSI/NCSL  Z540. 3-2006  or  the  ISO- 17025  be  adopted  by  the  RCC/SMSG  for  the  EO/IR 
laboratory  community.  Alternatively,  and  as  a  minimum,  evaluation  criteria  based  on  ANSI 
Z540. 1-1994  (R2002),  as  presented  in  Chapter  2  ,  could  be  used,  but  because  of  the  breadth  and 
diversity  of  facilities/organizations  that  might  follow  this  program,  the  more  contemporary 
standards  would  likely  be  more  palatable.  The  criteria  below  reflect  modifications  made  by  the 
evaluators  after  the  three  evaluations.  The  modifications  were  made  to  delete  ambiguous, 
duplicative,  or  irrelevant  requirements,  and  improve  clarity  and  readability. 

The  recommendation  of  the  participants  is  that  the  evaluation  criteria  in  Chapter  2  be 
reviewed  and  additional  testing  of  those  criteria  by  multiple  facilities  occurs  before  finalizing  the 
criteria.  A  multi-facility  test  of  the  process  and  criteria  would  follow  the  same  path  as  the  RCS 
certification  program,  in  which  three  very  different  facilities  were  employed  to  test  the  process 
and  criteria.  Once  more  broadly  tested  criteria  have  been  refined,  those  criteria  should  be 
presented  to  the  EO/IR  community,  writ  large,  along  with  the  background  as  to  how  those 
criteria  were  evolved,  so  that  broad  consensus  can  be  reached. 

Compliance  with  the  evaluation  criteria  presented  below  does  not  imply  compliance  with 
ANSI  Z-540  or  any  other  quality  standard,  although  the  intent  is  to  satisfy  the  basic  requirements 
for  a  quality  measurement  program. 

3.2  Construct  of  the  Code 

The  construct/format  of  the  modified  code  is  shown  in  Table  3-1.  Table  3-2  shows  the 
modified  Z-540- 1994-1 -based  evaluation  criteria  for  EO/IR  Measurement 
Facilities/Organizations  (MFOs),  and  follows  the  construct  in  Table  3-1. 
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TABLE  3-1.  CONSTRUCT  OF  THE  CODE 

sXX 

nXXX 

pX 

pnXX 

Section  number 
(from  the  unmodified 
criteria) 

Criteria  number, 
starting  with  nOOl  as 
the  first  criteria  in 
Section  1  and  going 
to  nil 5  for  the  last 
criteria  in  Section  21 

Priority  level 
(where  pi  is  the 
highest  priority  and 
p3  is  the  lowest 
priority 

(See  Chapter  2) 

Priority  number 
within  the  associated 
Section  (sXX) 

In  Table  3-2,  the  number  in  the  criteria  column  (e.g.,  9.1:  Does  the  laboratory...)  is  the 
criteria  number  from  the  original  Z-540-based  criteria  presented  in  Chanter  2? 

TABLE  3-2.  MODIFIED  Z-540-1994-1-BASED  EVALUATION  CRITERIA  FOR 
EO/IR  MEASUREMENT  FACILITIES/ORGANIZATIONS 

Code  Construct 

Criteria 

s09-n043-pl-pn01 

9.1:  Does  the  laboratory  have  an  established  and  documented  program  to 
ensure  its  calibration  program  supports  the  stated  calibration  performance 
of  its  measurement  system(s)? 

s09-n044-pl-pn02 

9.2:  Is  the  laboratory  calibration  process  traceable  to  a  published  (AMTA, 
IEEE,  etc.)  calibration  item  standard  or  other  national  standard  (when 
developed)? 

s09-n045-pl-pn03 

9.3:  Does  the  laboratory  measurement  and  calibration  program  include  a 
schedule  for  calibration  intervals  for  each  piece  of  test  equipment  used  to 
assess  system  performance? 

s09-n046-p  1  -pn04 

9.4:  Is  there  documentary  evidence  the  schedule  (9.3)  is  being  followed? 

s09-n047-pl-pn05 

9.6:  Does  the  laboratory  have  a  documented  program  to  monitor  test 
equipment  and/or  calibration  items  between  scheduled  calibrations  and 
verifications? 

s09-n048-pl-pn06 

9.7:  When  the  laboratory  participates  in  inter-laboratory  comparisons 
(See  5.6),  is  the  laboratory's  system  configuration  employed  in  those 
comparisons  clearly  documented? 

s09-n049-pl-pn07 

On-site  Review:  For  the  primary  calibration  items,  are  samples  of 
"acceptable"  calibrations  or  verification  records  available? 

sl0-n050-pl-pn08 

10.1-a:  Does  the  laboratory  have  documented  system-wide  calibration 
procedures? 

sl0-n051-pl-pn09 

10.2-a:  Are  separate  calibration  and  measurement  procedures  for  each 
major  measurement  system  or  for  each  type  of  measurement  conducted 
available? 
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Code  Construct 

Criteria 

sl0-n052-pl-pnl0 

10.2-b:  Are  the  calibration  measurement  procedures  consistent  with  the 
accuracy  required  for  each  type  of  calibration  measurement  (as  defined  in 
the  uncertainty  analysis  in  Section  19)? 

sl0-n053-pl-pnll 

10.3:  Are  the  procedures  for  deriving  or  assigning  values  for  a  measured 
target  through  the  use  of  transfer  standards  documented? 

sl0-n054-pl-pnl2 

10.4:  Are  system-wide  calibrations  scheduled  on  an  appropriate  cycle  for 
the  measurement  system?  Is  this  cycle  defined?  Is  there  evidence  they  are 
being  performed  when  scheduled?  Are  the  results  of  system-wide 
calibrations  included  or  referenced  in  this  section?  Are  these  calibrations 
provided  to  the  customer  as  part  of  their  normal  data  report? 

sl0-n055-pl-pnl3 

10.5:  If  sampling  is  included  as  a  part  of  calibration  or  measurements,  is 
documentation  available  that  describes  the  procedures  and  applied 
statistical  techniques  employed  (detailed  data  processing  implementations 
may  be  provided/discussed  in  Section  18,  if  applicable)? 

sl0-n056-pl-pnl4 

10.6:  Do  the  results  of  acceptable  system-wide  calibrations  support  the 
uncertainty  bounds  established  in  Section  19? 

sl0-n057-pl-pnl5 

10.7-a:  For  computer  software  used  to  acquire  and/or  store  the  raw  data, 
are  user  manual(s)  available  either  as  automated  help  files,  or  as  separate 
document(s),  which  describe  the  correct  operation  and  use  of  this 
software?  If  the  manual  is  not  available  on-line,  is  the  physical  location  of 
the  documentation  clearly  identified? 

sl0-n058-pl-pnl6 

10.7-b2  Is  there  a  database  or  other  appropriate  tracking  list  of  the 
computer  program(s)  that  are  used  in  the  end-to-end  data  acquisition, 
transmission,  and  processing  steps? 

sl0-n059-pl-pnl7 

10.7-b3  For  computer  software  used  to  process  the  raw  data  files  into  final 
data  products,  are  appropriate  user’s  manual  and/or  help  files  available? 

sl0-n060-pl-pnl8 

10.7-cl  Is  an  assigned  computer  support  and/or  network  administrator 
available  to  maintain  computer  equipment  used  in  the  laboratory  data 
acquisition  and  processing? 

sl0-n061-pl-pnl9 

10.7-c2  Is  there  a  diagram  of  the  local  area  network  within  the  laboratory, 
if  applicable? 

sl0-n062-pl-pn20 

10.7-d:  Are  computers  used  to  monitor  critical  environmental  and/or 
operating  procedures  in  order  to  maintain  the  stated  calibration  accuracy 
and/or  stay  within  the  published  calibration  uncertainty  of  Section  19?  If 
so,  are  the  operators  alerted  if  the  conditions  are  unacceptable? 

s!0-n063-pl-pn21 

10.7-e:  Are  there  procedures  that  ensure  security  of  the  data  to  prevent  the 
unauthorized  or  inadvertent  access,  manipulation,  or  destruction  of 
computer  data  or  records?  (Note  that  an  approved  network  AIS  or  SPP 
plan  will  satisfy  this  requirement.  If  the  AIS/SPP  is  not  electronically 
available  in  the  laboratory  book,  its  physical  location  must  be  precisely 
identified.) 
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s  1 0-n064-p  1  -pn22 

On-site  Review:  Request  the  laboratory  manager  perform  a  system-wide 
calibration.  Verify  the  documented  procedures  are  followed  by  laboratory 
technicians. 

sl0-n065-pl-pn23 

On-site  Review:  Compare  the  results  of  the  system-wide  calibration  with 
the  historical  records  found  in  Sections  12,  13  and/or  19.  Are  the  results 
within  the  stated  calibration  uncertainty  limits? 

sl0-n066-pl-pn24 

On-site  Review:  Ask  a  technician  if  the  observed  documented  procedures 
are  up  to  date.  Ask  when  the  procedures  were  last  reviewed  for 
completeness  and  accuracy. 

sll-n067-pl-pn25 

11.1:  Does  the  laboratory  have  a  system  for  uniquely  identifying  primary 
calibration  items? 

sll-n068-pl-pn26 

1 1.2:  Does  the  laboratory  have  a  system  for  assessing  the  calibration 
items  upon  initial  receipt?  Are  visual  inspection(s)  performed  on  the 
calibration  items  as  part  of  the  calibration  process?  Is  there  a  stated 
procedure  for  reacting  to  items  that  are  out  of  tolerance? 

sll-n069-pl-pn27 

11.3:  Does  the  laboratory  have  documented  procedures  and  appropriate 
facilities  to  avoid  deterioration  or  damage  to  calibration  items  during 
storage,  handling,  preparation,  and  calibration? 

sll-n070-pl-pn28 

1 1.4  Is  the  exact  location  of  the  calibration  item  standard  identified  when 
not  in  use? 

sll-n071-pl-pn29 

1 1.6:  If  the  primary  calibration  items  employed  by  the  laboratory  are  used 
in  any  capacity  other  than  to  ensure  the  performance  of  the  measurement 
system(s)  do  such  uses  damage  the  calibration  items? 

sll-n072-pl-pn30 

On-site  Review:  Perform  a  visual  inspection  of  the  laboratory’s 
calibration  items  to  determine  the  conditions,  procedures,  and  facilities 
assessed  in  11.1  through  11.6  are  as  stated. 

sl8-n098-pl-pn31 

18.1:  Does  the  laboratory  have  all  data  processing  procedures 
documented? 

sl8-n099-pl-pn32 

18.2:  Have  the  data  processing  modules  been  validated  with  a  standard 
data  set  to  ensure  the  correctness  of  the  algorithms  employed? 

sl8-nl00-pl-pn33 

18.3:  Are  the  results  of  data  processing  compared  against  known 
quantities  or  standards  to  verify  accuracy  of  the  measurement  and  data 
processing  process? 

sl8-nl01-pl-pn34 

On-site  Review:  Is  there  a  procedure  to  collect,  determine  and  validate  all 
customer  data  processing  requirements,  to  include  such  key  items  as 
specific  formats,  parameters  and  quantities?  Is  there  a  procedure  for 
translating  user  requirements  into  the  specific  procedures  to  collect  the 
data  identified  by  the  customer? 

s!8-nl02-pl-pn35 

On-site  Review:  Are  procedures  in  place  to  coordinate  target  readiness 
and  state  with  data  collectors  and  data  processors  so  that  the  target 
configuration  is  captured  throughout  the  data  collection  and  processing 
chain? 
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sl8-nl03-pl-pn36 

On-site  Review:  Are  procedures  in  place  for  validating  the  data  in  (near) 
real-time  to  ensure  the  proper  data  is  being  collected? 

sl8-nl04-pl-pn37 

On-site  Review:  Ask  a  laboratory  technician  to  process  a  file  from  a 
standard  data  set  and  compare  the  results  to  those  known  to  be  correct. 

sl8-nl05-pl-pn38 

On-site  Review:  Ask  the  laboratory  technical  manager  how  new  or 
improved  data  processing  algorithms  are  validated. 

sl8-nl06-pl-pn39 

On-site  Review:  Review  the  results  of  any  inter-laboratory  data 
processing  comparisons.  Are  these  results  available  to  laboratory 
customers? 

sl9-nl07-pl-pn40 

19.1  Does  the  laboratory  have  a  policy  for  the  development  and  use  of  a 
calibration  uncertainty  analysis?  (Alternate  location  for  section  5.2-u.) 

sl9-nl08-pl-pn41 

19.2  For  the  primary  calibration  standard(s)  used  by  the  laboratory,  is 
there  a  nominal  measurement  with  estimated  uncertainties,  in  either 
graphical  or  tabular  form  for  that  standard? 

sl9-nl09-pl-pn42 

19.3  Is  the  calibration  uncertainty  shown  in  19.2  consistent  with  stated 
calibration  capabilities  of  the  laboratory  in  other  Laboratory  Book 
sections?  (Note:  If  the  laboratory’s  stated  calibration  accuracy  is  less  than 
the  accuracy  shown  in  this  nominal  measurement,  those  capabilities  must 
be  changed  to  match  those  obtained  in  the  section  19.2  measurements.) 

sl9-nll0-pl-pn43 

19.4  Are  the  results  of  the  uncertainty  analysis  for  primary  calibration 
reflected  in  the  day-to-day  measure  of  calibration  quality  discussed  in 
Section  10?  (Note:  In  other  words,  are  typical  uncertainties  used  as  a 
quality  check  for  daily  calibration  measurements?) 

sl9-nlll-pl-pn44 

On-site  Review:  Ask  the  technicians  to  show  (in  tabular  or  graphical 
form)  the  predicted  uncertainty  of  the  primary  calibration  standard.  Can 
the  technicians  find  the  information? 

sl9-nll2-pl-pn45 

On-site  Review:  Do  the  technicians  use  the  primary  calibration  target 
uncertainty  bars  to  bracket  acceptable  (daily)  calibration  as  a  quality 
check? 

sl9-nll3-pl-pn46 

On-site  Review:  Are  the  actions  taken  by  the  technicians  when  calibration 
measurements  fall  outside  expectations  consistent  with  the  stated 
laboratory  policy  on  exceptions  to  approved  calibration  process  (See 
Sections  5  and  10)? 

s05-n015-p2-pn01 

5.2-a:  Is  there  a  management  quality  statement  here  or  in  Section  1? 

s05-n016-p2-pn02 

5.2-c:  Are  the  roles  and  responsibilities  of  management,  technical 
operations,  support  services,  and  quality  systems  identified  here  or  in 
Section  4? 

s05-n017-p2-pn03 

5.2-d:  Is  there  a  description  of  how  documents  are  controlled  and 
referenced? 
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s05-n018-p2-pn04 

5.2-i:  Does  the  laboratory  have  a  process  to  assess  whether  it  has  the 
technical  capabilities  and  resources  (including  schedule  time  and 
consumables)  before  scheduling  new  work? 

s05-n019-p2-pn05 

5.2-o:  Is  there  a  clearly  established  policy  and  process  when  measurement 
discrepancies  are  seen  or  departures  from  documented  policies  and 
procedures  occur? 

s05 -n020-p2-pn06 

5.2-t:  Is  the  policy  for  establishing  and  changing  calibration  intervals 
documented? 

s05-n021-p2-pn07 

5.3:  Are  the  laboratory's  internal  and  external  audit  processes  documented 
(including  reviewers  names,  date  audit  started/completed,  complete  report, 
and  final  scores)?  Have  previous  audit  deficiencies  been  corrected  by  the 
next  annual  audit? 

s05-n022-p2-pn08 

5.4:  Was  a  management  quality  audit  performed  within  the  previous  12 
months? 

s05-n023-p2-pn09 

5.6:  Does  the  laboratory  have  a  process  for  tracking  and  monitoring  the 
quality  of  the  primary  calibration  data  for  calibration  items  used  in  the 
normal  measurement  process?  Is  there  a  day-to  day  pass/fail  check  criteria 
established  for  calibration  measurements?  Have  the  results  been  verified, 
either  through  repeated  measurements,  inter-laboratory  comparisons  or 
both? 

s05-n024-p2-pn 1 0 

On-site  Review:  Ask  laboratory  technicians  what  the  pass/fail  criteria  for 
calibration  are. 

s05-n025-p2-pnll 

On-site  Review:  Ask  laboratory  technicians  what  is  done  if  calibrations 
depart  expectations. 

s08-n035-p2-pnl2 

8.1:  Does  the  laboratory  have  a  list,  database,  or  other  log  or 
configuration  control  method  for  identifying  and  tracking  the  main 
laboratory  equipment  systems  and  subsystems?  Are  there  omissions? 

s08-n036-p2-pnl3 

8.2-a:  For  laboratory  equipment  identified  in  8.1,  is  a  maintenance 
schedule  identified  for  periodic  or  preventative  maintenance? 

s08-n037-p2-pnl4 

8.2-b:  Is  there  any  evidence  the  equipment  maintenance  schedules  are 
being  followed  (i.e.,  last  .check-on.  date,  maintenance  logs,  etc.)? 

s08-n038-p2-pnl5 

8.2-c:  For  major  laboratory  systems  or  subsystems,  are  maintenance 
procedures  clearly  identified? 

s08-n039-p2-pnl6 

On-site  Review:  Select,  at  random,  five  pieces  of  equipment  from  the  log 
of  8.1  above.  Can  the  pieces  of  equipment  be  located? 

s08-n040-p2-pnl7 

On-site  Review:  Ask  to  see  the  maintenance  procedures  (if  applicable)  for 
those  pieces  of  equipment.  Are  they  up  to  date? 

s08-n041-p2-pn!8 

On-site  Review:  Ask  to  see  the  reference  manuals  for  the  five  pieces  of 
equipment  selected.  Are  the  manuals  readily  available?  Were  those 
manuals  found  in  the  locations  stipulated  in  the  Laboratory  Book? 
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On-site  Review:  For  the  five  pieces  of  equipment  selected,  is  there  a 
historical  log  kept  on  damage,  repair,  or  re-calibration  (if  appropriate)? 

13.1:  Is  there  a  description  and  purpose  for  each  type  of  certificate  or 
report  employed  or  delivered  by  the  laboratory  (e.g.,  measurement  data 
and  test  summary  reports)? 

13.2- a:  Does  each  certificate  or  report  include:  a  title  (e.g.,  Measurement 
Report);  name  and  address  of  the  facility,  points  of  contacts,  phone/fax 
numbers  and  location(s)  of  the  test(s);  name,  address  and  points  of  contact 
of  the  customer;  description  and  unambiguous  identification  of  the 
measured  item  (including  its  configuration,  as  appropriate); 
characterization  and  condition  of  the  measured  item;  date(s)  and  times(s) 
of  the  measurement(s);  identification  of  the  calibration  procedure(s)  used 
or  unambiguous  description  of  non-standard  procedures  employed; 
reference  to  sampling  and/or  data  processing  procedures,  where  relevant; 
any  deviations  from,  additions  to  or  exclusions  from  documented 
operating  procedures,  and  other  information  relevant  to  the 
measurement(s),  such  as  environmental  conditions  or  failures/anomalies 
encountered  during  the  measurement(s)? 

13.2- b:  Does  each  certificate  or  report  include  measurements, 
examinations  and  derived  results,  supported  by  tables,  graphs,  sketches 
and  photographs  (and/or  digital  representations  of  such),  as  appropriate  for 
the  measurements  conducted? 


sl3-n078-p2-pn23 

13.2-c:  Does  each  certificate  or  report  include  a  statement  of  the  estimated 
uncertainty  of  the  primary  calibration  results  and  the  traceability  of  the 
measurements? 

sl3-n079-p2-pn24 

13.2-d:  Does  each  certificate  or  report  include  a  signature  and  title  of  the 
person(s)  accepting  responsibility  for  the  content  of  the  certificate  or 
report,  and  the  date  of  issuance? 

sl3-n080-p2-pn25 

13.2-e:  Does  each  certificate  or  report  include  a  statement  regarding  any 
limitations  on  the  use  or  interpretation  of  the  measurement  data? 

sl3-n081-p2-pn26 

13.3:  Are  portions  of  the  measurement  data  products  performed  by  sub¬ 
contractors?  If  so,  are  such  portions  clearly  identified? 

sl3-n082-p2-pn27 

13.4:  Is  there  a  procedure  for  amending  a  report?  Is  there  a  procedure  for 
notifying  customers  promptly,  in  writing,  of  any  event  such  as  discovery 
of  defective  calibration  equipment  that  may  cast  doubt  on  the  validity  of 
the  results  provided  in  any  given  report  or  certificate,  including  the 
magnitude  of  the  errors  that  may  exist? 

sl3-n083-p2-pn28 

13.6:  Is  there  a  documented  procedure  for  transmitting  reports  or 
certificates  to  customers  or  third  party  recipients  such  that  confidentiality 
is  preserved?  Note:  a  hand  receipt,  computer  receipt,  registered  mail 
receipt,  or  other  appropriate  tracking  system  may  be  used  to  fulfill  this 
requirement. 

s08-n042-p2-pnl9 

sl3-n075-p2-pn20 

sl3-n076-p2-pn21 


s 1 3-n077-p2-pn22 
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sl7-n094-p2-pn29 

17.1:  Does  the  laboratory  have  a  documented  policy  on  inter-laboratory 
comparisons? 

sl7-n095-p2-pn30 

17.2:  Does  the  laboratory  participate  in  an  inter-laboratory  calibration 
comparison  program  at  the  national  level? 

sl7-n096-p2-pn31 

17.3:  Are  the  results  of  the  inter-laboratory  calibration  comparisons  of 
Section  17.2  available  for  customer  review? 

sl7-n097-p2-pn32 

17.4:  Are  the  results  of  the  inter-laboratory  comparisons  used  to  evaluate 
and  improve  the  laboratory's  operation? 

s21-nll4-p2-pn33 

20-1:  Is  there  a  description  or  summary  of  a  three-year  look-ahead  into 
future  plans,  upgrades,  and  quality  or  laboratory  improvements? 

s21-nll5-p2-pn34 

20-2:  Is  there  a  roadmap  formatted  to  summarize  the  three-year  plan. 

(Note  that  the  roadmaps  can  be  longer  than  3  years,  but  must  as  a 
minimum  look-ahead  of  3  years). 

s01-n001-p3-pn01 

1.1:  Is  the  host  organization  identified?  Is  the  endorsement  signed  by  a 
high-level  manager  or  director?  (Recommended  to  be  at  least  two  levels 
of  management  above  the  laboratory  quality  manager.) 

sO  1  -n002-p3-pn02 

1.3:  Is  there  a  clear  commitment  to  quality  in  the  introduction  and 
endorsement? 

s01-n003-p3-pn03 

On-site  Review:  Ask  to  meet  the  endorser.  Is  the  endorser  clearly 
committed  to  the  quality  process? 

s02-n004-p3-pn04 

2.2:  Are  all  references  unambiguous  enough  to  identify  documents? 

s02-n005 -p3-pn05 

2.3:  Are  the  documents  cross-referenced  or  linked  appropriately? 

s02-n006-p3-pn06 

On-site  Review:  Ask  to  see  5  documents  at  random  in  this  list. 

s03-n007 -p3-pn07 

3.1:  Is  there  an  appropriate  glossary? 

s03-n008-p3-pn08 

3.2:  Do  acronyms  (jargon)  occur  that  are  not  defined  (immediately)  in  that 
section  or  listed  here? 

s04-n009-p3-pn09 

4.2:  Does  the  laboratory  identify  a  laboratory  technical  manager,  a 
laboratory  quality  manager  and  alternates  in  case  of  absence? 

s04-n010-p3-pnl0 

4.4:  Does  the  laboratory  identify  the  management  chain  from  the 
technician  level  to  the  highest  level  of  the  organization? 

s04-n011-p3-pnll 

4.6:  Is  there  a  clear  policy  on  non-disclosure  and/or  company  proprietary 
data  or  information? 

s04-n012-p3-pnl2 

4.7:  If  laboratory  operators,  engineers,  and/or  technicians  are  from  a 
different  organization  or  company  than  the  laboratory  owners,  are  the 
respective  roles  and  responsibilities  of  the  two  organizations  and  their 
interfaces  clearly  spelled  out? 

s04-n013-p3-pnl3 

4.8:  Are  personnel  qualifications  available  or  referenced? 

s04-n014-p3-pn!4 

On-site  Review:  Is  the  laboratory  supervision  done  by  person(s)  familiar 
with  the  calibration  methods  and  procedures,  the  objective  of  the 
calibration  and  how  to  assess  the  results? 
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s06-n026-p3-pnl5 

6.1:  Does  the  laboratory  employ  an  appropriate  mix  of  technician, 
engineering  and  management  personnel  with  appropriate  experience  to 
perform  their  assigned  functions?  (Alternate  location  for  5.2-e) 

s06-n027-p3-pnl6 

6.2:  Does  the  laboratory  have  a  formal  training  program  and  if  so,  are 
records  kept  up  to  date?  Is  the  training  offered  sufficient  to  ensure  or 
maintain  technical  competence? 

s06-n028-p3-pnl7 

6.3:  Are  the  training  records  available  to  laboratory  management? 

s06-n029-p3-pnl8 

On-site  Review:  Ask  to  locate  the  personnel  training  records  of  6.3. 

s07-n030-p3-pnl9 

7.1:  Does  the  laboratory  control  and  monitor  critical  or  appropriate 
environmental  parameters  which  may  affect  the  proper  performance  and 
calibrations? 

s07-n031-p3-pn20 

7.2:  Are  the  nominal  measurement  environmental  factors  specified  for 
acceptable  operations  (e.g.,  allowable  wind  speeds,  temperature  variations, 
etc)? 

s07-n032-p3-pn21 

7.3:  Are  appropriate  environmental  parameters  recorded  as  a  part  of  the 
test  process? 

s07 -n033-p3-pn22 

On-site  Review:  Are  the  environmental  factors  identified  being  recorded 
in  a  log  or  with  the  data?  For  environmental  measurements  being 
recorded,  review  such  data  and  operations  with  laboratory  technicians. 

s07 -n034-p3-pn23 

On-site  Review:  Is  there  any  measurement  data  showing  system  stability 
or  repeatability  versus  stated  control  values  for  environmental  variations? 
(In  other  words,  are  the  factors  in  7.2  derived  based  on  data  and  if  so,  what 
data?) 

sl2-n073-p3-pn24 

12.1:  Does  the  laboratory  have  well-documented  procedures  for  creating 
and  maintaining  records  of  laboratory  operations,  to  include  records  of 
calibration,  measurements  and  system  configuration? 

s  1 2-n07 4-p3-pn25 

12.2:  Are  these  records  indexed  or  stored  so  they  are  easily  located  and 
accessible  to  those  with  a  need  to  review  them. 

sl4-n084-p3-pn26 

14.1:  If  the  laboratory  employs  any  subcontracting  in  the  performance  of 
measurements,  is  there  a  documented  procedure  or  policy  that  stipulates 
and  verifies  such  subcontractors  adhere  to  the  provisions  of  an  appropriate 
quality  management  system?  Does  the  laboratory  maintain  records  of  the 
work  accomplished  by  subcontractors? 

sl4-n085-p3-pn27 

14.2:  If  the  laboratory  employs  a  contractor  to  perform  measurements, 
does  the  contract  or  other  operational  document  bind  the  contractor  to  the 
appropriate  procedures? 

s!4-n086-p3-pn28 

On-site  Review:  Ask  the  laboratory  manager  if  subcontractors/contractors 
support  or  perform  measurements.  If  applicable,  review  the  contract(s)  to 
verify  flow-down  of  the  quality  system  requirements. 
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sl5-n087-p3-pn29 

15.1:  If  the  laboratory  uses  outside  services  or  suppliers  which  might 
impact  the  quality  and  accuracy  of  measurements,  are  there  documented 
policies  that  ensure  the  level  of  quality  of  the  laboratory's  calibrations  and 
measurement  products? 

sl5-n088-p3-pn30 

15.3:  Where  applicable,  are  there  procedures  for  maintaining  records 
associated  with  outside  services  or  suppliers? 

sl5-n089-p3-pn31 

On-site  Review:  If  outside  suppliers  or  services  are  used  at  the  laboratory 
under  review,  ask  to  see  the  associated  records. 

sl6-n090-p3-pn32 

16.1:  Does  the  laboratory  have  documented  procedures  for  the  resolution 
of  customer  complaints? 

sl6-n091-p3-pn33 

16.2:  Are  records  maintained  of  all  complaints  and  of  the  actions  taken  by 
the  laboratory  to  resolve  the  complaints? 

sl6-n092-p3-pn34 

On-site  Review:  Examine  records  of  a  random  complaint  (if  applicable)  to 
ensure  that  the  complaints  were  reviewed  and  that  appropriate  action  was 
taken  and  recorded. 

sl6-n093-p3-pn35 

On-site  Review:  Ask  the  laboratory  quality  manager  if  and  how  they  are 
involved  in  the  complaint  resolution  process.  Does  the  laboratory  quality 
manager  review  all  customer  complaints  for  indications  that  current 
policies  and  procedures  are  in  need  of  review? 
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CHAPTER  4 


SURVEY  INSTRUCTIONS  AND  SURVEY  FORM  (NOV  2006) 

4.1  Introduction 

As  part  of  the  RCC  effort  to  serve  the  technical  and  operational  needs  of  member  test 
ranges  or  facilities,  the  SMSG  was  formed  to  document  signature  measurement  standards  across 
the  electromagnetic  spectrum,  and  specifically  in  the  following  areas:  radar,  millimeter  wave 
(MW),  infrared  (IR),  laser,  visible,  ultraviolet  (UV),  seismic,  acoustic,  and  magnetic.  The 
unclassified  survey  was  distributed  as  an  initial  effort  to  collect  feedbacks  from  member  EO/IR 
ranges  or  facilities  for  subjects  and  contents  related  to  EO/IR  measurements  that  should  be 
included  in  the  SMSG  draft.  The  instructions  used  for  the  survey  and  the  survey  form  are  at 
paragraph  4.3  and  paragraph  4.4,  respectively. 

4.2  Objective 

The  objectives  of  the  survey  were  to: 

a.  Collect  feedbacks  on  technical  and  operational  needs  among  member  EO/IR  ranges; 

b.  Exchange  ideas  on  common  measurement  procedures  that  can  be  standardized; 

c.  Leam  common  error  or  uncertainty  analysis  methodology  and  format  currently  used; 

d.  Find  common  interests  in  hardware  or  instrumentation  to  be  included  in  the  draft; 

e.  Find  common  documentation  and  data  report  format  preferences 

f.  Elect  candidate  applications  or  measurements  suitable  for  range  demonstration 
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4.3  Specific  Instructions  used  for  the  Survey 


The  survey  is  in  MS  Word  2003  format.  It  makes  use  of  the  Forms  tools.  In  all  cases, 
please  check  all  that  apply.  The  comment  fields  for  the  ‘Other’  boxes  are  of  unlimited  length 
and  will  wrap  around  if  you  need  additional  space.  As  there  are  drop  down  boxes,  it  is  intend  to 
be  filled  in  electronically.  If  needed,  the  form  can  be  printed  out  and  the  answers  hand  written. 
For  the  drop  down  boxes,  please  use  the  same  answers  as  listed  in  the  drop  down  box. 

Please  return  completed  survey  by  Dec  15  by  one  of  the  following: 


Email:  Jeffrey.Burks@wpafb.af.mil  or  Scott.Milster@atk.com 


Fax:  (937)  656-7074  (Jeff  Burks) 
Mail:  Jeff  Burks  or 
AFRL/SNS 
2591  K  St 
WPAFB  OH  45433 


or  (937)  431-1402  (Scott  Milster) 
Scott  Milster 
ATK  Mission  Research 
3975  Research  Blvd 
Dayton,  OH  45430 


For  question  8,  if  you  need  additional  subsections  for  different  instruments  it  is  easy  to 
expand  these.  First,  turn  off  the  ‘Protect  Document’  feature.  Do  this  by  going  to  the  Tools  menu 
and  clicking  on  ‘Protect  Document’,  or  if  you  have  the  ‘Forms  Toolbar’  displayed  click  on  the 
‘Protect  Form’  button  which  looks  like  a  lock.  There  is  no  password  set  for  the  form.  At  this 
point,  you  can  edit  the  document  just  like  any  Word  document,  so  you  can  cut  and  paste  as 
needed  to  expand  the  form. 


Once  you  are  done  editing,  start  enforcing  the  protection  by  either  clicking  on  the  button 
‘Yes,  Start  Enforcing  Protection’  to  the  right,  or  clicking  on  the  ‘Protect  Form’  button  on  the 
‘Forms  Toolbar’.  Make  sure  the  ‘Editing  Restrictions’  allow  ‘Filling  in  Forms’.  This  is  the 
initial  setting  so  you  should  not  have  to  change  it.  If  the  ‘Password’  box  appears,  just  leave  it 
blank  and  click  Okay.  This  will  allow  you  to  tab  through  the  form  fields. 


Note:  The  format  is  only  there  to  make  it  easier  for  you  to  respond.  Our  main  objective  is  the 
information,  not  the  format!  Electronic  responses  in  English  are  preferred,  but  we  will  accept 
typewritten,  faxed,  handwritten,  or  any  other  way  you  choose  to  convey  the  information. 


Thank  you  for  your  time.  We  plan  to  compile  the  responses  and  create  a  catalog  in 
addition  to  the  standard  procedures.  We  need  everyone’s  help  to  make  it  as  comprehensive  as 
possible.  We  look  forward  to  your  response. 


Sincerely, 


Jeff  Burks 
AFRL/SNS 
937  255  9205 


Scott  Milster 

ATK  Mission  Research 

(937)  429  9261  extension  156 
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4.4  Survey  Form 


RCC/SMSG  EO/IR  SURVEY  -  NOVEMBER  2006 

•  Contact  Information 
Name: 

Phone  Number: 

E-Mail: 

•  What  types  of  targets  do  you  collect  EO/IR  information  on? 

Coupons:  Scale  Models: 

Full  size/actual  targets: 

Other  (  ): 

•  What  type  of  tests  do  you  conduct?  Check  all  that  apply. 

i  I  Static  EH  Wind  Tunnel  EH  Free  Flight  EH  Sled  EH  Wire  guided 

EH  Other  _ 

I  |  Air  to  air: 

Sensor  aircraft: 

Maximum  altitude  Maximum  test  duration 

I  I  Air  to  gnd  EH  Gnd  to  air  EH  Gnd  to  Gnd  EH  Air  to  ship 

•  What  type  of  measurements  do  you  collect  of  the  target?  Check  all  that  apply. 

EH  Radiometric  EH  Images  EH  Spectral  EH  Spectral  Images  EH  Polarization 

EH  Reflectance  EH  Transmittance  EH  BRDF  EH  DHR 

I  I  Temperature  range: 

I  I  Thermocouple,  type  EH  RTD,  type 

I  I  Other:  _ 

•  What  types  of  calibration  sources  do  you  have?  Check  all  that  apply. 

H  Blackbody:  Highest  temp.  ,  Fowest  temp.  ,  Size  of  largest  BB 
n  Common  Source  EH  Spectral  lamps 

I  I  Other 

•  What  type  of  calibrations  do  you  do?  Check  all  that  apply. 

I  I  NIST  Traceable  O  Absolute  O  Relative 

•  What  ancillary  data  do  you  collect?  Check  all  that  apply. 

I  I  Time  &  Space  Position  Information  (TiSPI)  of  Target 
Weather/ Atmospheric: 

I  |  Wind  speed  Q  Dew  point  Q  Humidity  Q  Precipitation 

□  CO2  EH  Visibility  EH  Temperature  EH  Pressure 

EH  Other  atmospheric  data: 

EH  Other  Ancillary  Data: 


Organization/Range : 
FAX  Number: 
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•  What  are  the  characteristics  of  your  instruments?  (Repeat  as  needed  for  each  camera, 
radiometer,  reflectometer,  monochromator,  spectrometer  or  other  instrument  that  you 
would  like  us  to  know  about.  See  the  instructions  if  you  run  out  of  room.) 

o  Instrument  type: 

Manufacturer/Model: 

Bands  or  spectral  range: 

Frame  Rate:  Aperture  size:  f/#: 

Lens(es): 

Year  Purchased:  Software  used:  _ 

How  do  you  calibrate  this  instrument?  Q  Blackbody,  Q  Common  Source,  Q  Internal 

calibration,  O  Spectral  lamps,  O  None  needed,  O  Other 

Comments: 

o  Instrument  type: 

Manufacturer/Model : 

Bands  or  spectral  range: 

Frame  Rate:  Aperture  size:  f/#: 

Lens(es): 

Year  Purchased:  Software  used: 

How  do  you  calibrate  this  instrument?  Q  Blackbody,  Q  Common  Source,  O  Internal 

calibration,  O  Spectral  lamps,  O  None  needed,  O  Other 

Comments: 

o  Instrument  type: 

Manufacturer/Model: 

Bands  or  spectral  range: 

Frame  Rate:  Aperture  size:  f/#: 

Lens(es): 

Year  Purchased:  Software  used:  _ 

How  do  you  calibrate  this  instrument?  Q  Blackbody,  Q  Common  Source,  O  Internal 

calibration,  O  Spectral  lamps,  O  None  needed,  O  Other 

Comments: 

o  Instrument  type: 

Manufacturer/Model : 

Bands  or  spectral  range: 

Frame  Rate:  Aperture  size:  f/#: 

Lens(es): 

Year  Purchased:  Software  used: 

How  do  you  calibrate  this  instrument?  Q  Blackbody,  Q  Common  Source,  Q  Internal 

calibration,  O  Spectral  lamps,  O  None  needed,  O  Other 

Comments: 
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•  Do  you  have  standard  report  templates?  Q  Yes  O  No 

Are  you  willing  to  provide  a  copy  of  your  templates  or  sample  reports?  O  Yes  O  No 
Please  attach  if  Yes. 

•  Do  you  have  documented  procedures  for  Error  or  Uncertainty  Analysis?  O  Ycs|  |  No 
Are  you  willing  to  provide  a  copy  of  your  documented  procedures?  O  Yes  I  I  No 
Please  attach  if  Yes. 

•  What  specific  items  are  included  in  your  reports? 

Experimental  data:  O  Raw,  O  Calibrated 

i  |  Data  Collection  Procedures,  O  Calibration  Procedures,  O  Calibration  Results 
□  Modeled  data  and  procedures 
|~|  Theoretical  predictions  and  derivations 
I  |  Uncertainty  analysis 
I  I  Other: 

•  Your  answers  to  this  survey  are: 

\~\  Classified,  Classification  Level  Other: 

I  I  Unclassified  O  FOUO 

Has  the  following  distribution  condition: 

!  |  A.  Approved  for  public  release;  distribution  is  unlimited. 

I  I  B.  Distribution  authorized  to  U.S.  Government  Agencies  only  Reason: 

,  Date  of  Determination:  .  Other  requests  for  this 

document  shall  be  referred  to  the  controlling  DoD  office  in  Item  1  above. 

I  |  C.  Distribution  authorized  to  U.S.  Government  Agencies  and  their  contractors 

Reason:  ,  Date  of  Determination:  .  Other  requests  for  this 

document  shall  be  referred  to  the  controlling  DoD  office  in  Item  1  above. 

I  I  D.  Distribution  authorized  to  the  Department  of  Defense  and  U.S.  DoD  contractors 
only  Reason:  ,  Date  of  Determination:  .  Other  requests 

shall  be  referred  to  the  controlling  DoD  office  in  Item  1  above. 

I  |  E.  Distribution  authorized  to  DoD  Components  only  Reason:  , 

Date  of  Determination:  .  Other  requests  shall  be  referred  to  the  controlling 

DoD  office  in  Item  1  above. 

I  I  F.  Further  dissemination  only  as  directed  by  the  controlling  DoD  office  in  Item  1 
above  Date  of  Determination:  or  higher  DoD  authority. 

I  I  X.  Distribution  authorized  to  U.S.  Government  Agencies  and  private  individuals  or 
enterprises  eligible  to  obtain  export-controlled  technical  data  in  accordance  with 
reference  (c)  Date  of  Determination:  .  Controlling  DoD  office  is  listed  in  Item 

1  above. 

I  I  Export  Control  Warning:  WARNING  -  This  document  contains  technical  data  whose 
export  is  restricted  by  the  Arms  Export  Control  Act  (Title  22,  U.S.C.,  Sec  2751,  et. 
seq.)  or  the  Export  Administration  Act  of  1979,  as  amended.  Title  50,  U.S.C.,  App. 
2401  et.  seq.  Violations  of  these  export  laws  are  subject  to  severe  criminal  penalties. 
Disseminate  in  accordance  with  provisions  of  DoD  Directive  5230.25. 
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CHAPTER  5 

SMSG  EO/IR  SURVEY  RESULTS  &  INTERPRETATION 

This  chapter  is  fundamentally  based  on  a  document  entitled,  SMSG  Project  Final  Report 
Draft  Version  1,  delivered  12  October  2007  to  the  SMSG-18  task  lead,  Mr.  Jeffrey  W.  Burks 
(EO/IR  Laboratory,  Code  SNS  (now  Code  RYS)  at  the  Air  Force  Research  Laboratory  (AFRL), 
Wright-Patterson  AFB .  The  document  was  prepared  under  contract  by  EO/IR  Remote  Sensing 
Group,  ATK  Space  Systems  Inc.  That  document  and  this  chapter  summarize  the  status  of  the 
Task  SMSG-18  project  covering  the  period  from  2  June  2006  to  30  September  2007. 

The  original  project  scope  covered  by  the  aforementioned  report  by  ATK  was  to 
document  standards  for  characterizing  the  measurement  capabilities  of  EO/IR  measurement 
systems  and  to  help  in  establishing  a  demonstration  program  that  verifies  and  quantifies  the 
improvement  in  measurement  capability  that  will  result  from  implementing  documented 
standards.  Due  to  technical  reasons,  part  of  the  original  project  scope  was  modified  or  dropped 
during  the  early  stage.  For  example,  it  was  originally  intended  to  define,  in  draft  of  the  standard, 
levels  of  information  contents  that  are  synonymous  with  levels  of  knowledge  as  well  as  the 
requirements  to  meet  these  levels.  That  requirement  was  later  dropped;  however,  as  much  work 
was  done  on  that  part  of  the  effort,  the  draft  results  are  presented  in  this  SMSG-18  Final  Report 
in  Chapter  6,  Draft  Document  Standards  for  Characterizing  the  Measurement  Capabilities  of 
EO/IR  Measurement  Systems. 

Another  change  was  to  switch  the  basis  of  that  report  to  a  document  then  titled 
Development  ofEO  Standard  Processes  for  Application  (EOS PA),  which  has  been  included  in 
this  SMSG-18  Final  Report  as  Chapter  7,  Draft  Handbook  for  Electro-Optical  Measurements. 
Other  modifications  on  the  original  project  scope  are  reflected  in  later  sections. 

5.1  Introduction 

This  chapter  summarizes  the  status  of  the  SMSG  project  covering  the  period  from  June  2, 
2006  to  September  30,  2007.  As  part  of  the  RCC  effort  to  serve  the  technical  and  operational 
needs  of  member  test  ranges  or  facilities,  the  SMSG  was  formed  to  document  signature 
measurement  standards  across  the  electromagnetic  (EM)  spectrum,  specifically  in  the  following 
areas:  radar,  millimeter  wave  (MMW),  infrared  (IR),  laser,  visible,  ultraviolet  (UV),  seismic, 
acoustic,  and  magnetic.  The  reported  project,  officially  titled  Development  of  EO  Standard 
Processes  for  Application  (EOSPA),  is  limited  to  the  EO/IR  portion  of  the  EM  spectrum. 

5.2  Original  Project  Scope  and  Modifications 

The  original  project  scope  covered  by  the  aforementioned  ATK  report  was  to  document 
standards  for  characterizing  the  measurement  capabilities  of  EO/IR  measurement  systems  and  to 
help  in  establishing  a  demonstration  program  that  verifies  and  quantifies  the  improvement  in 
measurement  capability  that  will  result  from  implementing  documented  standards.  Due  to 
technical  reasons,  part  of  the  original  project  scope  was  modified  or  dropped  in  the  early  stage. 
For  example,  it  was  originally  intended  to  define,  in  draft  of  the  standard,  levels  of  information 
contents  that  are  synonymous  with  levels  of  knowledge  as  well  as  the  requirements  to  meet  these 
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levels.  That  requirement  was  later  dropped;  however,  as  much  work  was  done  on  that  part  of  the 
effort,  the  draft  results  are  presented  in  this  SMSG-18  Final  Report  in  Chapter  6,  Draft 
Document  Standards  for  Characterizing  the  Measurement  Capabilities  ofEO/OR  Measurement 
Systems. 

Another  change  was  to  switch  the  basis  of  that  report  to  a  document  then  titled 
Development  ofEO  Standard  Processes  for  Application  (EOS PA),  which  has  been  included  in 
this  SMSG-18  Final  Report  as  Chapter  7  with  its  titled  changed  to  Draft  Handbook  for  Electro- 
Optical  Measurements.  References  herein  to  EOSPA  refer  to  the  material  contained  in  Chapter 
7.  Other  modifications  on  the  original  project  scope  are  reflected  in  later  sections. 

5.3  Sub  Tasks 

Ten  sub  tasks  were  established  within  the  original  project  scope.  Individual  sub  tasks 
were  detailed  in  the  original  SOW  (Statement  of  Work). 

5.4  Implementation  and  Results 

5.4. 1  Task  1  (EOSPA  Draft).  It  was  defined  in  Task  1  that  a  draft  of  EOSPA  be  prepared  and 
distributed  by  MRC.  The  draft  was  to  be  based  on  the  sample  measurement  procedures  and 
reports  from  member  EO/IR  ranges  or  facilities.  Early  communications  among  member  EO/IR 
ranges  or  facilities  in  attempt  to  collect  or  exchange  such  samples  did  not  produce  good  results. 

In  part,  difficulties  were  due  to  the  following: 

a.  The  majority  of  the  facilities  and  programs  were  classified  which  made  open 
communication  difficult. 

b.  Significant  effort  from  the  member  EO/IR  ranges  was  needed  to  provide  the 
requested  information. 

c.  No  agreed-upon  sample  procedures/reports  were  distributed  as  examples  of  the 
information  requested  and  many  labs  were  unwilling  to  share  their  documents. 

d.  The  SMSG  project  relied  heavily  on  the  format  of  electronic  communications  (e- 
mails  and  phone  calls)  and  did  not  plan  or  allocate  budget  for  travel  to  allow  person- 
to-person  meetings. 

Under  such  circumstances,  the  first  version  of  the  draft  of  EOSPA  in  the  format  of  a 
Microsoft  Word  document  was  completed  on  October  10,  2006.  However,  no  sample  procedures 
or  reports  from  member  EO/IR  ranges  were  included  since  none  was  available  at  the  time. 

5.4.2  Task  2  (Distribution  and  Survey).  As  required  in  Task  2,  MRC  distributed  the  first  draft 
of  EOSPA  with  survey/questionnaire  to  the  organizations  identified  by  the  RCC  to  solicit 
feedback  from  the  same  organizations.  The  survey/questionnaire  included  questions  that  will 
help  in  identifying  an  initial  set  of  ranges  that  are  willing  to  demonstrate  the  standard  procedures 
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to  be  documented  in  the  EOSPA  draft.  Eleven  feedbacks  were  collected  after  three  rounds  of 
survey  distributions.  The  process  included: 

a.  First  Distribution.  The  first  version  of  the  draft  of  EOSPA  in  the  format  of  a  Word 
document  with  survey  and  instructions  attached  was  distributed  to  about  10  to  15 
attendees  during  a  SMSG  meeting  held  in  October  17-18,  2006.  None  of  the 
organizations  surveyed  sent  back  feedback. 

b.  Second  Distribution.  To  make  responding  to  the  survey  easier,  a  second  version  in 
the  form  of  an  Excel  spreadsheet  was  completed  on  November  27,  2006.  The  survey 
was  distributed  again  to  the  known  EO/IR  ranges  and  SMSG  members.  Feedback 
from  four  groups  was  collected  in  this  round  but  none  of  them  included  sample 
procedures  or  reports.  One  of  the  respondents  was  not  willing  to  share  its  responses 
with  the  broader  community,  so  those  responses  were  not  included  in  the  tally. 

c.  Third  Distribution.  To  solicit  more  feedbacks,  the  SMSG  modified  the  survey  and 
redistributed  it  via  e-mail  during  January  10-11,  2007  to  about  110  contacts  who  were 
involved  in  EO/IR  measurements.  Follow-up  telephone  calls  were  made  to  all  the 
contacts.  Feedback  from  seven  groups  was  collected  but  none  of  them  included 
sample  procedures  or  reports. 

d.  Final  Sample  Request.  In  an  effort  to  collect  sample  procedures  and  reports,  e-mail 
requests  and  phone  calls  were  made  during  March  8-19,  2007  to  the  organizations 
who  responded  to  the  survey.  This  effort  did  not  produce  any  results.  Some 
organizations  informed  MRC  that  the  procedures  and  reports  are  for  their  internal  use 
only.  Their  management  did  not  approve  the  request.  Other  organizations  simply 
said  that  they  did  not  have  any. 

5.4.3  Task  3  (First  Release).  After  several  round  of  revisions,  MRC  complied  and  distributed  a 
first  draft  of  the  EOSPA  as  reported  in  paragraph  5.4.1  and  paragraph  5.4.2.  However,  the 
document  was  not  refined  to  incorporate  any  sample  procedures  or  reports  since  none  was 
collected. 

5.4.4  Task  4  (Demo  Ranges).  None  of  the  EO/IR  ranges  volunteered  to  demonstrate  the 
procedures  to  be  documented  in  the  draft.  Not  completed. 

5.4.5  Task  5  (Training).  Not  completed. 

5.4.6  Task  6  (Final  Report).  This  document  serves  as  the  final  report. 

5.4.7  Task  7  (Application).  As  an  example  application,  the  main  instrumentation  and 
measurement  procedures  used  in  the  EO/IR  lab  were  summarized  in  a  report. 

5.4.8  Added  Task  (RF  Z540  equivalent).  As  an  additional  task,  ATK- MR  was  tasked  to 
produce  a  document  similar  to  the  RF  Z540  but  more  tailored  to  the  EO/IR  community,  which  is 
included  in  this  Task  SMSG-8  Final  Report  as  Chapter  3,  Modified  ANSI  Z-540.1-1994-based 
Evaluation  Criteria  for  use  by  EO/IR  Facilities. 
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5.5  Major  Project  Timeline 

a.  May  19,  2006:  MRC  received  SOW  for  the  SMSG  project  via  e-mail. 

b.  Jun.  2,  2006:  Held  SMSG  SP50  proposal  review/delegation  meeting  at  MRC. 

c.  Aug.  11,  2006:  Completed  the  first  outline  for  the  draft  of  EOSPA;  distributed  to  NIST 
for  comments. 

d.  Sept.  19,  2006:  Completed  the  first  draft  of  EOSPA. 

e.  Oct.  10,  2006:  Modified  the  first  draft  of  EOSPA  based  on  comments  from  NIST. 

f.  Oct.  17-18,  2006:  Distributed  a  draft  of  EOSPA  with  the  first  survey /questionnaire  to 
attendees  of  the  SMSG  meeting. 

g.  Nov.  27,  2006:  Compiled  and  distributed  the  second  SMSG  survey  with  instructions 

h.  Jan.  9-11,  2007:  Compiled  a  SMSA  contact  list  (Excel  spreadsheet);  distributed  the 
SMSG  survey  via  e-mails;  made  phone  calls  to  most  of  the  SMSG  members  on  the  list. 

i.  Jan  15,  2007:  Wrote  a  summary  on  the  status  of  the  SMSG  project. 

j.  Feb.  1,  2007:  Updated  SMSG  contact  list  (Excel  spreadsheet)  to  include  only  EO/IR 
members;  made  phone  calls  or  send  e-mails  to  the  EO/IR  members  on  the  new  list  and 
emphasized  Question  No. 9  in  the  Survey. 

k  Mar.  8-19,  2007:  Requested  sample  procedures  or  reports  again  from  the  EO/IR  ranges 
who  responded  to  the  surveys  via  e-mails  and  phone  calls. 

l.  Sept.  14,  2007:  Draft  of  the  final  project  report 

m.  Sept.  30,  2007 :  Final  project  report 

5.6  Survey  Results 

The  information  obtained  from  the  collected  survey  feedback  is  summarized  in  this 

section. 

5.6.1  Organizations  and  POC.  The  names  of  POC  (Point-of-Contact)  who  replied  to  the  survey 
and  the  organizations  with  which  they  were  affiliated  were  originally  listed  in  Table  5-1  of  the 
original  ATK/MRC  report.  To  make  this  Task  SMSG- 18  Final  Report  publicly  releasable,  that 
information  was  redacted.  Contact  the  Range  Commanders  Council  Secretariat  to  learn  if  that 
information  can  be  disclosed  through  other  channels.  In  the  table  that  follows,  and  in  subsequent 
references  in  later  sections  to  the  participating  organizations,  generic  placeholders  are  employed. 
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TABLE  5-1.  PARTICIPANT  PLACEHOLDER 

No. 

Participant  Placeholder 

1 

P-1 

2 

P-2 

3 

P-3 

4 

P-4 

5 

P-5 

6 

P-6 

7 

P-7 

8 

P-8 

9 

P-9 

10 

P-A 

11 

P-B 

5.6.2  Types  of  Targets.  Among  the  organizations  who  used  targets,  only  P-1  considered 
background  in  their  tests;  P-3  also  tested  the  material  of  their  target;  P-8  performed  component 
tests  in  a  fixture  as  well. 

a.  Coupons.  Organizations  that  used  coupons  are  listed  in  Table  5-2. 


TABLE  5-2.  ORGANIZATIONS  USED  COUPONS 

Organization 

Coupons 

Frequency 

No.  of  Targets/Yr 

P-2 

Common 

7 

P-4 

Occasionally 

3 

P-7 

Occasionally 

3 

P-8 

Common 

7 

P-A 

Occasionally 

3 

P-B 

Common 

7 

b.  Scaled  Models.  Organizations  used  scaled  models  are  listed  in  Table  5-3. 


TABLE  5-3.  ORGANIZATIONS  USED  SCALED  MODELS 

Organization 

Scaled  Models 

Frequency 

Targets/Yr 

Type 

P-1 

Occasionally 

3 

n/a 

P-3 

Rarely 

1 

Aircraft,  rockets 

P-7 

Rarely 

1 

n/a 

P-8 

Rarely 

1 

n/a 

P-A 

Occasionally 

3 

n/a 

P-B 

Occasionally 

3 

n/a 
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c.  Full-Size  Target.  Organizations  that  used  full-size  targets  are  listed  in  Table  5-4. 


TABLE  5-4.  ORGANIZATIONS  USED  FULL-SIZE  TARGETS 

Org. 

Full  Size  or  Actual  Target 

Frequency 

Targets/ 

Yr 

Type 

Type 

P-1 

Common 

7 

n/a 

n/a 

P-2 

Common 

7 

n/a 

n/a 

P-3 

Common 

7 

Plumes,  hardbodies,  muzzle,  flash, 
high  altitude  events,  launches, 
intercepts 

Plumes,  hardbodies,  muzzle,  flash, 
high  altitude  events,  launches, 
intercepts 

P-4 

Common 

7 

n/a 

n/a 

P-6 

Common 

7 

n/a 

n/a 

P-7 

Often 

6 

n/a 

n/a 

P-8 

Common 

7 

n/a 

n/a 

P-9 

Common 

7 

n/a 

n/a 

P-B 

Rarely 

Yes 

n/a 

n/a 

5.6.3  Types  of  Tests.  The  types  of  tests  conducted  by  the  organizations  are  summarized  in 
Table  5-5.  P-2  reported  a  maximum  test  altitude  of  500m  and  maximum  test  duration  of  3min. 
P-A  uses  an  indoor  range  for  their  tests.  P-8  also  performs  other  type  of  tests. 


TABLE  5-5.  TYPES  OF  TESTS 

O 

era 

Static 

Wind 

Tunnel 

Free 

Flight 

Sled 

Wire 

Guided 

Land  to 
Ship 

Air  to 
Ship 

Air  to 
Gnd 

Gnd 

to 

Air 

Gnd 

to 

Gnd 

Tower 

/TT 

Moving  Target/ 
Sensor 

P-1 

No 

No 

No 

No 

No 

No 

No 

No 

Yes 

Yes 

No 

No 

P-2 

Yes 

No 

Yes 

No 

No 

Yes 

No 

No 

No 

No 

No 

No 

P-3 

Yes 

Yes 

Yes 

Yes 

Yes 

No 

Yes 

Yes 

Yes 

Yes 

No 

No 

P-4 

No 

No 

No 

No 

No 

No 

No 

No 

Yes 

Yes 

Yes 

No 

P-5 

No 

No 

No 

No 

No 

No 

No 

No 

No 

No 

No 

No 

P-6 

Yes 

No 

No 

No 

No 

No 

Yes 

Yes 

Yes 

Yes 

No 

Yes 

P-7 

Yes 

No 

No 

No 

No 

No 

No 

No 

Yes 

Yes 

No 

No 

P-8 

Yes 

No 

No 

No 

No 

No 

No 

No 

No 

No 

No 

No 

P-9 

n/a 

n/a 

n/a 

n/a 

n/a 

n/a 

n/a 

n/a 

n/a 

n/a 

n/a 

n/a 

P-A 

No 

No 

No 

No 

No 

No 

No 

No 

No 

No 

No 

No 

P-B 

Yes 

No 

No 

No 

No 

No 

No 

No 

Yes 

Yes 

No 

No 
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5.6.4  Types  of  Measurements.  The  type  of  measurements  performed  by  the  organizations  is 
summarized  in  Table  5-6.  Measurement  temperature  range  and  the  sensors  used  to  monitor  the 
temperature  are  shown  in  Table  5-7. 


TABLE  5-6.  TYPES  OF  MEASUREMENT 

Organization 

Radiometric 

Images 

Spectral 

Spectral 

Images 

Polarization 

Reflectance 

Transmittance 

P-1 

Yes 

Yes 

Yes 

No 

No 

Yes 

No 

P-2 

Yes 

Yes 

Yes 

No 

No 

Yes 

Yes 

P-3 

Yes 

Yes 

Yes 

No 

No 

No 

No 

P-4 

Yes 

Yes 

Yes 

Yes 

No 

Yes 

No 

P-5 

No 

No 

No 

No 

No 

No 

No 

P-6 

Yes 

Yes 

Yes 

Yes 

No 

No 

No 

P-7 

Yes 

Yes 

Yes 

No 

No 

No 

No 

P-8 

Yes 

Yes 

No 

No 

No 

Yes 

Yes 

P-9 

No 

Yes 

Yes 

Yes 

No 

No 

No 

P-A 

Yes 

Yes 

Yes 

No 

No 

Yes 

Yes 

P-B 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

TABLE  5-7.  TEMPERATURE  RANGE  AND  SENSORS 

Organization 

Temperature 

Range 

Thermocouple/T  ypes 

RTD 

Target 

Illumination 

Min 

(°C) 

Max 

(°C) 

Yes/No 

K 

T 

J 

E 

I- 

buttons 

Yes/No 

Type 

P-1 

n/a 

n/a 

Yes 

Yes 

No 

No 

No 

Yes 

No 

n/a 

No 

P-2 

200 

3000 

No 

No 

No 

No 

No 

No 

No 

n/a 

No 

P-3 

n/a 

n/a 

No 

No 

No 

No 

No 

No 

No 

n/a 

No 

P-4 

n/a 

n/a 

Yes 

Yes 

Yes 

No 

No 

No 

No 

n/a 

No 

P-5 

n/a 

n/a 

No 

No 

No 

No 

No 

No 

No 

n/a 

No 

P-6 

-20 

50 

No 

No 

No 

No 

No 

No 

No 

n/a 

Yes 

P-7 

n/a 

n/a 

No 

No 

No 

No 

No 

No 

No 

n/a 

No 

P-8 

n/a 

n/a 

No 

No 

No 

No 

No 

No 

No 

n/a 

No 

P-9 

n/a 

n/a 

No 

No 

No 

No 

No 

No 

No 

n/a 

No 

P-A 

n/a 

n/a 

Yes 

No 

No 

No 

Yes 

No 

Yes 

100D 

No 

P-B 

-40 

1200 

Yes 

Yes 

Yes 

Yes 

No 

No 

Yes 

n/a 

No 
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5.6.5  Calibration  Sources.  The  types  of  calibration  sources  used  by  the  organizations  are  listed 
in  Table  5-8. 


TABLE  5-8.  CALIBRATION  SOURCES 


TABLE  5-8.  CALIBRATION  SOURCES 

Org. 

Blackbody 

Common 

Source 

Spectral 

Lamps 

D2 

Lamps 

Reflectance 

Panels 

Other 

Sources 

Yes/ 

No 

Tr„in  (°C) 

Tmax 

(°C) 

Max 
Size  (in) 

Yes/No 

Yes/No 

Type 

Yes/No 

P-1 

1 

-40 

1400 

30  by  30 

No 

Yes 

- 

No 

No 

No 

P-2 

1 

-20 

550 

12  by  12 

No 

Yes 

n/a 

Yes 

No 

No 

P-3 

1 

10 

1200 

6 

Yes 

Yes 

n/a 

No 

No 

No 

P-4 

1 

-10 

600 

12  by  12 

Yes 

No 

n/a 

No 

No 

No 

P-5 

0 

n/a 

n/a 

n/a 

No 

No 

n/a 

No 

No 

No 

P-6 

1 

varies 

varies 

12 

No 

No 

n/a 

No 

Yes 

No 

P-7 

1 

Ambient 

600 

12 

No 

No 

n/a 

No 

No 

No 

P-8 

0 

n/a 

n/a 

n/a 

No 

No 

n/a 

No 

No 

Yes 

P-9 

0 

n/a 

n/a 

n/a 

No 

No 

n/a 

No 

No 

No 

P-A 

1 

-100 

1300 

6  by  6 

No 

No 

n/a 

No 

No 

No 

P-B 

1 

20 

1000 

4 

Yes 

Yes 

n/a 

No 

No 

No 

5.6.6  Calibration  Type.  The  calibration  type  is  summarized  in  Table  5-9. 


TABLE  5-9.  CALIBRATION  TYPE 

Organization 

NIST  Traceable 

Absolute 

Relative 

P-1 

Yes 

Yes 

Yes 

P-2 

Yes 

Yes 

Yes 

P-3 

Yes 

Yes 

Yes 

P-4 

Yes 

Yes 

Yes 

P-5 

Yes 

No 

No 

P-6 

Yes 

Yes 

Yes 

P-7 

Yes 

No 

Yes 

P-8 

No 

Yes 

Yes 

P-9 

No 

No 

No 

P-A 

Yes 

No 

No 

P-B 

Yes 

Yes 

Yes 
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5.6.7  Data  Collected.  The  types  of  data  collected  by  the  organizations  are  listed  in  Table  5-10, 
Table  5-11,  and  Table  5-12. 


TABLE  5-10.  ENVIRONMENTAL  DATA  COLLECTED 

Org. 

TSPI* 

Wind 

Speed 

Dew 

Point 

Humidity 

Precip 

Visibility 

Temp 

Pressure 

Other 

Atmospheric 

Data 

P-1 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

n/a 

P-2 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

n/a 

P-3 

No 

Yes 

Yes 

Yes 

No 

No 

Yes 

Yes 

n/a 

P-4 

Yes 

Yes 

Yes 

Yes 

No 

Yes 

Yes 

Yes 

n/a 

P-5 

No 

No 

No 

No 

No 

No 

No 

No 

n/a 

P-6 

Yes 

Yes 

Yes 

No 

Yes 

Yes 

No 

Yes 

n/a 

P-7 

Yes 

Yes 

Yes 

Yes 

Yes 

No 

No 

No 

n/a 

P-8 

Yes 

No 

No 

No 

No 

No 

No 

No 

n/a 

P-9 

Yes 

Yes 

No 

Yes 

No 

No 

Yes 

Yes 

n/a 

P-A 

No 

No 

No 

Yes 

No 

No 

Yes 

No 

n/a 

P-B 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

Variation  with 
altitude 

*  TSPI  =  time-space-position-information 

TABLE  5-11.  BACKGROUND  DATA  COLLECTED 

Organization 

Nephelometer 

Gnd  Temperature 

Soil  Moisture 
Content 

Cloud  Data 

P-1 

No 

No 

No 

No 

P-2 

Yes 

No 

No 

No 

P-3 

No 

No 

No 

No 

P-4 

No 

Yes 

Yes 

Yes 

P-5 

No 

No 

No 

No 

P-6 

No 

No 

No 

No 

P-7 

No 

No 

No 

No 

P-8 

No 

No 

No 

No 

P-9 

No 

No 

No 

No 

P-A 

No 

No 

No 

No 

P-B 

No 

No 

No 

No 
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TABLE  5-12.  SPECTRAL  OR  RADIOMETRIC  DATA  COLLECTED 

Organization 

co2 

IR  Radiation 

Solar 

Irradiance 

Ozone 

CN2 

Background 

Spectra 

P-1 

No 

Yes 

Yes 

No 

No 

No 

P-2 

No 

No 

Yes 

Yes 

No 

No 

P-3 

Yes 

No 

No 

Yes 

No 

No 

P-4 

No 

No 

No 

No 

No 

No 

P-5 

No 

No 

No 

No 

No 

No 

P-6 

Yes 

No 

Yes 

No 

Yes 

No 

P-7 

No 

No 

No 

No 

No 

No 

P-8 

No 

No 

No 

No 

No 

No 

P-9 

No 

No 

No 

No 

No 

No 

P-A 

No 

No 

No 

No 

No 

No 

P-B 

No 

No 

No 

No 

No 

Yes 

5.6.8  Report. 

a.  Templates  and  Samples.  Survey  results  on  the  usage  of  report  template  are  listed  in 
Table  5-13.  Three  organizations  said  that  they  have  samples  but  none  of  them  shared 
their  documents  with  MRC. 

b.  Error  or  Uncertainty  Analysis.  As  shown  in  Table  5-13,  only  three  organizations 
perform  error  or  uncertainty  analysis.  Although  P-B  had  samples,  they  did  not  share 
them. 


TABLE  5-13.  REPORT  TEMPLATES 
AND  SAMPLES 

Org. 

Report  Temp 

late 

Existing? 

Sample? 

Shared? 

P-1 

No 

No 

No 

P-2 

Yes 

No 

No 

P-3 

Yes 

No 

No 

P-4 

No 

Yes 

No 

P-5 

No 

No 

No 

P-6 

No 

No 

No 

P-7 

Yes 

Yes 

No 

P-8 

Yes 

No 

No 

P-9 

No 

No 

No 

P-A 

No 

No 

No 

P-B 

Yes 

Yes 

No 

TABLE  5-14.  ERROR  OR 
UNCERTAINTY  ANALYSIS 

Org. 

Error/Uncertainty  Analysis 

Doc 

Procedure? 

Sample? 

Shared? 

P-1 

No 

No 

No 

P-2 

No 

No 

No 

P-3 

Yes 

No 

No 

P-4 

No 

No 

No 

P-5 

Yes 

No 

No 

P-6 

No 

No 

No 

P-7 

No 

No 

No 

P-8 

No 

No 

No 

P-9 

No 

No 

No 

P-A 

No 

No 

No 

P-B 

Yes 

Yes 

No 
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c.  Data  Types,  Procedures,  and  Results.  Data  types,  procedures,  and  results  in  the  report 
are  listed  in  Table  5-15. 


TABLE  5-15.  DATA  TYPES,  PROCEDURES,  AND  RESULTS  REPORTED 

Org. 

Data  Type  Reported 

Procedures  Reported 

Results 

Reported 

Raw 

Calibrated 

Modeled 

Uncertainty 

Analysis 

Data 

Acquisition 

Calibration 

Data 

Modeling 

Calibration 

Results 

P-1 

Yes 

Yes 

No 

No 

Yes 

Yes 

No 

Yes 

P-2 

No 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

No 

P-3 

Yes 

Yes 

No 

No 

Yes 

Yes 

No 

Yes 

P-4 

No 

Yes 

Yes 

No 

Yes 

Yes 

Yes 

No 

P-5 

No 

No 

No 

No 

No 

No 

No 

No 

P-6 

Yes 

Yes 

Yes 

No 

Yes 

No 

Yes 

No 

P-7 

Yes 

Yes 

No 

No 

Yes 

Yes 

No 

Yes 

P-8 

Yes 

Yes 

Yes 

No 

Yes 

Yes 

Yes 

Yes 

P-9 

No 

No 

No 

No 

No 

No 

No 

No 

P-A 

No 

Yes 

No 

No 

Yes 

Yes 

No 

Yes 

P-B 

No 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

d.  Other  Contents.  Other  contents  in  report  are  listed  in  Table  5-16. 


TABLE  5-16.  OTHER  CONTENTS  IN  THE  REPORT 

Organization 

Imagery 

Database 

Theoretical 
Predictions  and 
Derivations 

Metadata 

Analysis  of 
Items  Under 
Test 

Sample 

History 

P-1 

No 

No 

No 

No 

No 

P-2 

No 

No 

No 

No 

No 

P-3 

No 

No 

No 

No 

No 

P-4 

Yes 

No 

No 

No 

No 

P-5 

No 

No 

No 

No 

No 

P-6 

No 

Yes 

No 

No 

No 

P-7 

No 

Yes 

Yes 

No 

No 

P-8 

No 

Yes 

No 

Yes 

No 

P-9 

No 

No 

No 

No 

No 

P-A 

No 

No 

No 

No 

No 

P-B 

No 

Yes 

No 

No 

Yes 
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5.6.9  Classification.  The  classification  assigned  by  each  organization  to  their  survey  reply  is 
listed  in  Table  5-17. 


TABLE  5-17.  SURVEY  CLASSIFICATION 

Survey  is 

Distribution 

Approved 
for  public 
release? 

Authorized  to  US  Government  Agencies 
and  their  contractors? 

Authorized  to  the 
DoD  and  their 
Contractors 

Export 

Control 

Warning 

Org. 

Unclass 

FOUO 

Yes/No 

Reason 

Date  of 
determination 

P-1 

Yes 

Yes 

No 

No 

n/a 

n/a 

No 

No 

P-2 

Yes 

Yes 

No 

No 

n/a 

n/a 

No 

No 

P-3 

Yes 

No 

Yes 

No 

n/a 

n/a 

No 

No 

P-4 

No 

Yes 

No 

Yes 

Critical 

technology 

Dec-06 

No 

No 

P-5 

Yes 

No 

Yes 

No 

n/a 

n/a 

No 

No 

P-6 

Yes 

Yes 

Yes 

Yes 

n/a 

n/a 

No 

No 

P-7 

Yes 

No 

Yes 

No 

n/a 

n/a 

No 

No 

P-8 

Yes 

No 

Yes 

No 

n/a 

n/a 

No 

No 

P-9 

Yes 

No 

No 

No 

n/a 

n/a 

Yes 

No 

P-A 

Yes 

No 

No 

No 

No 

No 

Yes 

No 

P-B 

No 

Yes 

No 

No 

No 

No 

No 

Yes 

5.6.10  Instruments.  Types  of  instruments  with  major  specifications,  the  suppliers,  and  owners 
are  listed  in  the  tables  in  this  section. 

a.  Colorimeter  or  Photometer. 


TABLE  5-18.  COLORIMETER  OR  PHOTOMETER 

Supplier 

Model 

Spectral  Range  (pm) 

Owner 

Qty  (unit) 

Photo  Research 

PR650 

0.38-0.78 

P-1 

1 

Photo  Research 

PR880 

Vis 

P-1 

1 

5-18 


RCC  Special  Report:  Development  Of  Electro-Optical  (EO)  Standard  Processes  for  Application,  November  2011 


b.  Radiometer. 

(1)  NIR  Band.  NIR  radiometers  with  major  characteristics  with  their  owners  are 
listed  in  Table  5-19.  Accessories  and  other  information  are  listed  in  Table  5-20. 


TABLE  5-19.  NIR  RADIOMETERS  AND  SPECIFICATIONS 

Supplier 

Radiometer 

Model 

Det 

Det  Size 

Spectral  Range 
(pm) 

Frame  Rate 
(Hz) 

Filter  Pass 
Bands  (pm) 

Owner 

Qty 

(Units) 

P-3 

UV  Rad  1 

CsTe 

n/a 

0.01  -  0.4 

<  20000 

0.2  -  0.4 

P-3 

1 

P-3 

UV  Rad  2 

CsTe 

n/a 

0.01  -  0.4 

<  20000 

0.2  -  0.4 

P-3 

1 

P-3 

UV  Rad  3 

MCP 

n/a 

0.01  -  0.4 

<  20000 

0.2  -  0.4 

P-3 

1 

P-3 

Rad  7 

InSb 

1  mm  chip 

Dual  band 

<  20000 

1.0 -5.0 

P-3 

1 

P-3 

Rad  8 

InSb 

1  mm  chip 

Dual  band 

<  20000 

1.0 -5.0 

P-3 

1 

P-3 

Rad  9 

InSb 

1  mm  chip 

Dual  band 

<  20000 

1.0 -5.0 

P-3 

1 

P-3 

Rad  10 

InSb 

1  mm  chip 

Dual  band 

<  20000 

1.0 -5.0 

P-3 

1 

P-3 

Rad  11 

InSb 

1  mm  chip 

Dual  band 

<  20000 

1.0 -5.0 

P-3 

1 

Xybion 

Xybion 

InSb 

1  mm  chip 

Dual  band 

n/a 

UV  -  Vis 

P-3 

1 

Xybion 

Xybion 

n/a 

n/a 

0.4  -  0.9 

Varies 

n/a 

P-4 

1 

Magnavox 

Magnavox 

n/a 

n/a 

Dual  band 

30 

n/a 

P-4 

1 

CEDIP 

CEDIP 

n/a 

n/a 

1.2-3 

Varies 

n/a 

P-4 

1 

TABLE  5-20.  NIR  RADIOMETER  ACCESSORIES  AND  OTHER  INFORMATION 


Supplier 

Radiometer 

Model 

Lenses/  FOV 

Year  of 
Purchase 

Bit  Depth 
(bit) 

Software 

Calibrated 

with 

Comment 

Owner 

P-3 

UV  Rad  1 

9  deg 

2000 

n/a 

Custom 

Spectral 

lamps 

CsTe  PMT,  Analog 

P-3 

P-3 

UV  Rad  2 

25  deg 

2001 

n/a 

Custom 

Spectral 

lamps 

CsTe  PMT,  Photon 
counter 

P-3 

P-3 

UV  Rad  3 

9,  25  deg 

2001 

n/a 

Custom 

Spectral 

lamps 

MCP  intensifier 

P-3 

P-3 

Rad  7 

2  and  7  deg 

2000 

n/a 

Custom  (Lab VIEW/ 
MATLAB) 

BB 

Single  1  mm  InSb  det 

P-3 

P-3 

Rad  8 

2  and  7  deg 

2000 

n/a 

Custom  (Lab VIEW/ 
MATLAB) 

BB 

Single  1  mm  InSb  det 

P-3 

P-3 

Rad  9 

2  and  7  deg 

2000 

n/a 

Custom  (Lab VIEW/ 
MATLAB) 

BB 

Single  1  mm  InSb  det 

P-3 

P-3 

Rad  10 

2  and  7  deg 

2000 

n/a 

Custom  (Lab VIEW/ 
MATLAB) 

BB 

Single  1  mm  InSb  det 

P-3 

P-3 

Rad  11 

2  and  7  deg 

2000 

n/a 

Custom  (Lab VIEW/ 
MATLAB) 

BB 

Single  1  mm  InSb  det 

P-3 

Xybion 

Xybion 

13mm-5080mm 

selectable, 

variable 

1997 

n/a 

HW  specific 

Spectral 

lamps 

Single  1mm  InSb 

P-3 

Xybion 

Xybion 

f/#  1.4 

2004 

n/a 

Custom 

Other 

Computer  controlled 
gain/exposure 

P-4 

Magnavox 

Magnavox 

Aperture  size 
12";  f/#  1.4 

1989 

n/a 

In-house 

BB 

n/a 

P-4 

CEDIP 

CEDIP 

n/a 

2004 

n/a 

Custom 

Other 

n/a 

P-4 
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(2)  Mid-wavelength  Infrared  (MWIR)  Band.  MWIR  radiometers  with  major 

characteristics  with  their  owners  are  listed  in  Table  5-21.  Other  parameters  and 
information  are  in  Table  5-22. 


TABLE  5-21.  MWIR  RADIOMETERS  AND  SPECIFICATIONS 

Supplier 

Model 

Det 

Array  Size 

Spectral 
Range  (pm) 

Frame  Rate 
(Hz) 

Owner 

Qty 

Amber 

Galileo 

InSb 

256  x  256 

3-5 

120 

P-1 

3 

Inframetrics 

760 

Scanning 

MTC 

257  x  207 

3-5 

30 

P-1 

2 

Electrophysics 

Jade  MW 

MCT 

320  x  240 

3-5 

200 

P-1 

1 

Electrophysics 

Emerald  MW 

MCT 

640x512 

3-5 

100 

P-1 

1 

Inframetrics 

PM  390 

PtSi 

n/a 

3-5 

30 

P-1 

1 

TABLE  5-22.  OTHER  PARAMETERS  AND  INFORMATION  FOR  THE  MWIR  RADIOMETERS 

Supplier 

Model 

Bit  Depth 
(bit) 

Software 

Calibrated 

with 

Owner 

Amber 

Galileo 

12 

Custom  Lab  VIEW 

Blackbodies 

P-1 

Inframetrics 

760 

8 

Thermagram  (old  DOS  version) 

Blackbodies 

P-1 

Electrophysics 

Jade  MW 

14 

Altair 

Blackbodies 

P-1 

Electrophysics 

Emerald  MW 

14 

Altair 

Blackbodies 

P-1 

Inframetrics 

PM  390 

8 

None 

None 

P-1 
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(3)  Long-wavelength  Infrared  (LWIR)  Radiometers.  LWIR  radiometers  with  major 
characteristics  and  their  owners  are  listed  in  Table  5-23.  Other  parameters  and 
information  are  listed  in  Table  5-24. 


TABLE  5-23  LWIR  RADIOMETERS  AND  SPECIFICATIONS 

Supplier 

Model 

Det 

Array  Size 

Spectral 
Range  (pm) 

Frame 

Rate 

(Hz) 

Owner 

Qty 

Indigo 

Merlin  Uncooled 

Microbolometer 

320  x  240 

7  -  14 

30 

P-1 

3 

Amber/Raytheon 

Radiance  1 

QWIP 

256  x  256 

8-9.2 

60 

P-1 

1 

Inframetrics 

760 

Scanning  MTC 

257  x  207 

8  -  12 

30 

P-1 

3 

Electrophysics 

Jade  LW 

MCT 

320  x  240 

7-9.2 

200 

P-1 

1 

FLIR 

ThermaCam  EX  320 

Microbolometer 

(320  x  240) 

(7.5  -  13) 

n/a 

P-1 

1 

TABLE  5-24.  OTHER  PARAMETERS  AND  INFORMATION  FOR  THE  LWIR  RADIOMETERS 

Supplier 

Model 

Bit  Depth 
(bit) 

Software 

Calibrated 

with 

Owner 

Indigo 

Merlin  Uncooled 

12 

Custom  Lab  VIEW 

Blackbodies 

P-1 

Amber/Raythe 

on 

Radiance  1 

12 

Rtools 

Blackbodies 

P-1 

Inframetrics 

760 

8 

Thermagram  (old  DOS  version) 

Blackbodies 

P-1 

Electrophysics 

Jade  LW 

14 

Altair 

Blackbodies 

P-1 

FLIR 

ThermaCam  EX 
320 

12 

ThermaCAM  QuickView 

Blackbodies 

P-1 

5-21 


RCC  Special  Report:  Development  Of  Electro-Optical  (EO)  Standard  Processes  for  Application,  November  2011 


c.  Hyperspectral  Radiometers.  Hyperspectral  radiometers  with  major  characteristics 
and  their  owners  are  listed  in  Table  5-25.  Other  parameters  and  information  are  listed 
in  Table  5-26. 


TABLE  5-25.  HYPERSPECTRAL  RADIOMETERS  AND  SPECIFICATIONS 

Supplier 

Radiometer 

Model 

Detector 

Model 

Det 

Array 

Size 

Spectral 

Range 

(pm) 

Frame 

Rate 

(Hz) 

Owner 

Qty 

(Unit) 

Pacific 

Advanced 

Technology 

LW IMSS 

Radiance 

1 

QWIP 

256  x 
256 

8-9.2 

n/a 

P-1 

1 

Pacific 

Advanced 

Technology 

MW  IMSS 

Galileo 

InSb 

256  x 
256 

3-5 

n/a 

P-1 

1 

n/a 

Custom 

n/a 

n/a 

n/a 

Dual  band 

n/a 

P-4 

1 

n/a 

Custom 

n/a 

n/a 

n/a 

0.4  -  0.77 

Varies 

P-4 

1 

TABLE  5-26.  OTHER  PARAMETERS  AND  INFORMATION  FOR  THE  HYPERSPECTRAL 
;  RADIOMETERS 

Supplier 

Radiometer 

Model 

Bit  Depth 
(bit) 

Software 

Calibrated 

with 

Year  of 
Purchase 

Comments 

Owner 

Pacific 

Advanced 

Technology 

LW  IMSS 

12 

n/a 

n/a 

n/a 

n/a 

P-1 

Pacific 

Advanced 

Technology 

MW  IMSS 

12 

n/a 

n/a 

n/a 

n/a 

P-1 

n/a 

Custom 

n/a 

Custom 

BB, 

Spectral 

lamps 

2005 

Hyperspectral  GSMS 
systems  collect 
2-Datacubes/sec 

P-4 

n/a 

Custom 

n/a 

Custom 

Field 

source 

2004 

M2 100 

P-4 
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d. 


Imaging  Radiometers.  Imaging  radiometers  with  major  characteristics  and  their 
owners  are  listed  in  Table  5-27.  Other  parameters  and  information  are  listed  in 


Table  5-28. 


TABLE  5-27.  IMAGING  RADIOMETERS  AND  SPECIFICATIONS 

Supplier 

Model 

Det 

Array  Size 

Spectral 

Range 

(pm) 

Frame 

Rate 

(Hz) 

Filter  Pass 
Bands 
(pm) 

Owner 

Qty 

(Unit) 

Indigo 

Merlin  NIR 

VisGaAs 

320  x  240 

0.6-  1.7 

<60 

n/a 

P-1 

1 

Indigo 

Phoenix 

n/a 

n/a 

other 

<60 

n/a 

P-2 

1 

FLIR 

SC6000 

n/a 

n/a 

other 

<60 

n/a 

P-2 

1 

FLIR 

SC2000 

n/a 

n/a 

6-40 

<60 

n/a 

P-2 

1 

Basler 

n/a 

n/a 

n/a 

0.4  -  0.77 

<500 

n/a 

P-3 

1 

FLIR 

SC6000 

InSb 

640x512 

3-6 

<  1500 

3-5 

P-3 

1 

FLIR 

SC6000 

InSb 

640x512 

3-6 

<  1500 

3.4 -4.1 

P-3 

1 

FLIR 

SC6000 

InGaAs 

640x512 

Dual  bands 

<  1500 

0.65  -  1.7 

P-3 

1 

Indigo 

Alpha  NIR 

InGaAs 

320  x  256 

Dual  bands 

<30 

0.9-  1.7 

P-3 

1 

Indigo 

Bolometer 

Bolometer 

320  x  240 

6-40 

<60 

8  -  12 

P-3 

1 

Indigo 

Merlin 

QWIP 

320  x  256 

6-40 

<60 

8-9 

P-3 

1 

n/a 

Phantom  9 

n/a 

1600  x  1200 

0.4-  1.7 

<  144000 

0.9-  1.7 

P-3 

1 

Photron 

Ultima  APXi2 
Fastcam 

n/a 

1024  x  1024 

Dual  bands 

<2000 

0.25  -  1.0 

P-3 

1 

Santa  Barbara 

FPA  SB#1 

InSb 

320  x  256 

Dual  bands 

<613 

1.0 -5.0 

P-3 

1 

Santa  Barbara 

FPA  SB#2 

InSb 

320  x  256 

Dual  bands 

<613 

1.0 -5.0 

P-3 

1 

Santa  Barbara 

FPA  SB#3 

InSb 

320  x  256 

Dual  bands 

<613 

1.0 -5.0 

P-3 

1 

Santa  Barbara 

FPA  SB#4 

InSb 

320  x  256 

Dual  bands 

<613 

1.0 -5.0 

P-3 

1 

Santa  Barbara 

FPA  SB#5A 

InSb 

640x512 

Dual  bands 

<  1500 

1.0 -5.0 

P-3 

1 

Santa  Barbara 

FPA  SB#5B 

InSb 

640x512 

Dual  bands 

<  1500 

1.0 -5.0 

P-3 

1 

Santa  Barbara 

FPA  SB#5C 

InSb 

640x512 

Dual  bands 

<  1500 

1.0 -5.0 

P-3 

1 

Santa  Barbara 

FPA  SBLW1 

QWIP 

320  x  256 

6-40 

<  1200 

8.0 -9.0 

P-3 

1 

Santa  Barbara 

FPA  SBLW2 

QWIP 

320  x  256 

6-40 

<  1200 

8.0 -9.0 

P-3 

1 

Sony 

Sony 

n/a 

630  x  480 

Dual  bands 

<30 

Vis  -  NIR 

P-3 

1 

Amber 

Galileo 

InSb 

256  x  256 

3-5 

120 

n/a 

P-1 

3 

Inframetrics 

760 

Scanning  MTC 

257  x  207 

3-5 

30 

n/a 

P-1 

2 

Electrophysics 

Jade  MW 

MCT 

320  x  240 

3-5 

200 

n/a 

P-1 

1 

Electrophysics 

Emerald  MW 

MCT 

640x512 

3-5 

100 

n/a 

P-1 

1 

Inframetrics 

PM  390 

PtSi 

n/a 

3-5 

30 

n/a 

P-1 

1 

Indigo 

Merlin  Uncooled 

Microbolometer 

320  x  240 

7-  14 

30 

n/a 

P-1 

3 

Amber/ 

Raytheon 

Radiance  1 

QWIP 

256  x  256 

8-9.2 

60 

n/a 

P-1 

1 

Inframetrics 

760 

Scanning  MTC 

257  x  207 

8  -  12 

30 

n/a 

P-1 

1 

Electrophysics 

Jade  LW 

MCT 

320  x  240 

7-9.2 

200 

n/a 

P-1 

1 

FLIR 

ThermaCam  EX 
320 

Microbolometer 

(320  x  240) 

(7.5  -  13) 

n/a 

n/a 

P-1 

1 
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TABLE  5-27.  IMAGING  RADIOMETERS  AND  SPECIFICATIONS 

Supplier 

Model 

Det 

Array  Size 

Spectral 

Range 

(pm) 

Frame 

Rate 

(Hz) 

Filter  Pass 
Bands 
(pm) 

Owner 

Qty 

(Unit) 

Agema 

Agema 

n/a 

n/a 

Dual  band 

25 

n/a 

P-4 

1 

FLIR 

n/a 

n/a 

n/a 

Dual  band 

Varied 

n/a 

P-7 

1 

Santa  Barbara 

FPA 

n/a 

n/a 

Dual  band 

Varied 

n/a 

P-7 

1 

Santa  Barbara 

FPA 

n/a 

n/a 

Dual  band 

Varied 

n/a 

P-7 

2 

Santa  Barbara 

FPA  LWIR 

n/a 

n/a 

6-40 

Varied 

n/a 

P-7 

1 

CEDIP 

Jade  VLWIR 

n/a 

320  x  240 

6-40 

200 
(full), 
600  (sub) 

8.0-  11.5 

P-A 

1 

Santa  Barbara 

FPA 

InSb 

640x512 

3-6 

157 

1-5.5 

P-A 

1 

Agema 

Agema  570 

Microbolometer 

320  x  240 

6-40 

60 

7.5-  13 

P-A 

1 

FLIR 

SC3000 

QWIP 

320  x  240 

Other 

900 

8-9 

P-A 

1 

FLIR 

ThermaCam  S65HS 

n/a 

320  x  240 

7.5  -  13 

60 

n/a 

P-B 

1 

Inframetrics 

ThermaCam  PM380 

n/a 

256  x  256 

3.4-5 

60 

n/a 

P-B 

1 

Inframetrics 

760 

n/a 

194x256 

Dual  band 

60 

n/a 

P-B 

1 
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TABLE  5-28.  OTHER  PARAMETERS  AND  INFORMATION  FOR  THE  IMAGING  RADIOMETERS 

Supplier 

Model 

Aperture/ 

Lenses/FOV 

Year 

of 

Purch 

Bit 

Depth 

(bit) 

Software 

Calibrated 

with 

Comments 

Owner 

Indigo 

Merlin  NIR 

n/a 

n/a 

14 

Rtools 

Integrating 

Sphere 

n/a 

P-1 

Indigo 

Phoenix 

lanos  16,  8,  4,  2  deg 

n/a 

n/a 

Rtools 

BB 

Superframing 

capability 

P-2 

FLIR 

SC6000 

lanos  16,  4,  2  deg 

2006 

n/a 

Rtools 

BB 

Superframing 

capability 

P-2 

FLIR 

SC2000 

24,  12,  7,  5  deg 

1997 

used 

n/a 

Rtools 

BB 

n/a 

P-2 

Basler 

n/a 

13  mm  -  5080  mm 
selectable 

2005 

n/a 

HW  specific 

Spectral 

lamps 

n/a 

P-3 

FLIR 

SC6000 

13  mm  -  5080  mm 
selectable,  0.11  -  52  deg 

2007 

n/a 

Rtools 

BB 

n/a 

P-3 

FLIR 

SC6000 

13  mm  -  5080  mm 
selectable,  0.11  -  52  deg 

2007 

n/a 

Rtools 

BB 

n/a 

P-3 

FLIR 

SC6000 

13  mm  -  5080  mm 
selectable,  0.11  -  52  deg 

2007 

n/a 

Rtools 

BB 

n/a 

P-3 

Indigo 

Alpha  NIR 

13  mm  -  5080  mm 
selectable,  variable 

2000 

n/a 

Camdaq/ 

Hyperview 

BB, 

spectral 

lamps 

n/a 

P-3 

Indigo 

Bolometer 

13,25,  50,  100  mm 
selectable 

2001 

n/a 

Camdaq/ 

Hyperview 

BB 

n/a 

P-3 

Indigo 

Merlin 

13  mm  -  5080  mm 
selectable,  0.88,  4.4,  34 
deg 

2002 

n/a 

Camdaq/ 

Hyperview 

BB 

n/a 

P-3 

n/a 

Phantom  9 

13  mm  -  5080  mm 
selectable,  variable 

2005 

n/a 

HW  specific 

Spectral 

lamps 

n/a 

P-3 

Photron 

Ultima 

APXi2 

Fastcam 

13  mm  -  5080  mm 
selectable,  variable 

2006 

n/a 

FastCam, 

custom 

Spectral 

lamps 

UV  enhanced 
intensified 

P-3 

Santa  Barbara 

FPA  SB#1 

13  mm  -  5080  mm 
selectable,  0.11  -  34  deg 

n/a 

n/a 

CamDaq/Rtools 

BB 

FPA 

P-3 

Santa  Barbara 

FPA  SB#2 

13  mm  -  5080  mm 
selectable,  0.11  -  34  deg 

n/a 

n/a 

CamDaq/ 

Rtools 

BB 

FPA 

P-3 

Santa  Barbara 

FPA  SB#3 

13  mm  -  5080  mm 
selectable,  0.11  -  34  deg 

n/a 

n/a 

CamDaq/Rtools 

BB 

FPA 

P-3 

Santa  Barbara 

FPA  SB#4 

13  mm  -  5080  mm 
selectable,  0.11  -  34  deg 

n/a 

n/a 

CamDaq/Rtools 

BB 

FPA 

P-3 

Santa  Barbara 

FPA  SB#5A 

13  mm  -  5080  mm 
selectable,  0.11  -  52  deg 

2004 

n/a 

CamDaq/Rtools 

AVinIR 

BB 

SBFP  MWIR 

P-3 

Santa  Barbara 

FPA  SB#5B 

13  mm  -  5080  mm 
selectable,  0.11  -  52  deg 

2004 

n/a 

CamDaq/Rtools 

AVinIR 

BB 

SBFP  MWIR 

P-3 

Santa  Barbara 

FPA  SB#5C 

13  mm  -  5080  mm 
selectable,  0.11  -  52  deg 

2004 

n/a 

CamDaq/Rtools 

AVinIR 

BB 

SBFP  MWIR 

P-3 

Santa  Barbara 

FPA  SBLW1 

13  mm  -  5080  mm 
selectable,  0.88  -  5.5  deg 

2006 

n/a 

WinIR 

BB 

SB  QWIP 

P-3 

Santa  Barbara 

FPA  SBLW2 

13  mm  -  5080  mm 
selectable,  0.88  -  5.5  deg 

2007 

n/a 

WinIR 

BB 

SB  QWIP 

P-3 
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TABLE  5-28.  OTHER  PARAMETERS  AND  INFORMATION  FOR  THE  IMAGING  RADIOMETERS 

Supplier 

Model 

Aperture/ 

Lenses/FOV 

Year 

of 

Purch 

Bit 

Depth 

(bit) 

Software 

Calibrated 

with 

Comments 

Owner 

Sony 

Sony 

13  mm  -  5080  mm 
selectable.  Variable 

2001 

n/a 

CamDaq/ 

Hyperview 

Spectral 

lamps 

Sony  630  x  480 

P-3 

Amber 

Galileo 

n/a 

n/a 

12 

Custom 

Lab  VIEW 

BB 

n/a 

P-1 

Inframetrics 

760 

n/a 

n/a 

8 

Thermagram 
(old  DOS  version) 

BB 

n/a 

P-1 

Electrophysics 

Jade  MW 

n/a 

n/a 

14 

Altair 

BB 

n/a 

P-1 

Electrophysics 

Emerald 

MW 

n/a 

n/a 

14 

Altair 

BB 

n/a 

P-1 

Inframetrics 

PM  390 

n/a 

n/a 

8 

None 

None 

n/a 

P-1 

Indigo 

Merlin 

Uncooled 

n/a 

n/a 

12 

Custom 

Lab  VIEW 

BB 

n/a 

P-1 

Amber/ 

Raytheon 

Radiance  1 

n/a 

n/a 

12 

Rtools 

BB 

n/a 

P-1 

Inframetrics 

760 

n/a 

n/a 

8 

Thermagram 
(old  DOS  version) 

BB 

n/a 

P-1 

Electrophysics 

Jade  LW 

n/a 

n/a 

14 

Altair 

BB 

n/a 

P-1 

FLIR 

ThermaCam 
EX  320 

n/a 

n/a 

12 

ThermaCAM 

QuickView 

BB 

n/a 

P-1 

Agema 

Agema 

n/a 

2004 

n/a 

Custom 

BB 

n/a 

P-4 

FLIR 

n/a 

Varied 

2005 

n/a 

Rtools/In-house 

BB 

n/a 

P-7 

Santa  Barbara 

FPA 

Varied 

2000 

n/a 

Rtools/In-house 

BB 

n/a 

P-7 

Santa  Barbara 

FPA 

Varied 

2002 

n/a 

Rtools/In-house 

BB 

n/a 

P-7 

Santa  Barbara 

FPA  LWIR 

Varied 

2002 

n/a 

Rtools/In-house 

BB 

n/a 

P-7 

CEDIP 

Jade  VLWIR 

25  mm;  f/#  2.0 

2006 

n/a 

Rtools/Altair 

BB 

Stirling  cooled; 
filter  wheel 

P-A 

Santa  Barbara 

FPA 

25,  50,  100  mm;  f/#  2.3 

2002 

n/a 

Rtools 

BB 

LN2  cooled; 
filter  wheel 

P-A 

Agema 

Agema  570 

24  x  18  deg,  12x9  deg 

1997 

n/a 

FSI 

BB 

No  filters 

P-A 

FLIR 

SC3000 

20  x  15  deg 

1999 

n/a 

FSI 

BB 

n/a 

P-A 

FLIR 

ThermaCam 

S65HS 

Aperture  Size  48  mm; 
120  mm,  36  mm,  f/# 

0.75 

2005 

n/a 

ThermaCam 

Researcher/ 

Quickview 

BB 

n/a 

P-B 

Inframetrics 

ThermaCam 

PM380 

8  and  16  deg  FOV;  f/1.5 

1996 

n/a 

Thermagram 

BB, 

internal 

calibration 

n/a 

P-B 

Inframetrics 

760 

0.5"  aperture;  15x20 
deg;  3X,  10X  zoom 

1990 

n/a 

Thermagram 

BB, 

internal 

calibration 

scanned  strip; 
vender 
calibration 

P-B 

e.  Spectral  Radiometer.  Spectral  radiometers  with  major  characteristics  and  their 
owners  are  listed  in  Table  5-29.  Other  parameters  and  information  are  listed  in 
Table  5-30. 
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TABLE  5-29.  SPECTRAL  RADIOMETER  AND  SPECIFICATIONS 

Supplier 

Model 

Det 

Spectral  Range 
(pm) 

Filter  Pass 
Bands  (pm) 

Frame  Rate  (Hz) 

Owne 

r 

Qty 

(Unit) 

ABB  Bomem 

FTIR  MR254 

InSb;  MCT 

1.3  -  13 

n/a 

10  Hz  @  1  wave- 
number 

P-1 

2 

Cl  Systems 

SR5000 

InSb;  MCT; 
SiPbS 

1.3  -  14.5; 

0.4  -  2.2 

n/a 

10  Hz  Spectral; 

1  kHz  Staring 

P-1 

2 

ABB  Bomem 

MR304SC 

n/a 

Dual  bands 

n/a 

<  82  spectra  per  sec 

P-2 

1 

SciTec 

8145 

n/a 

0.1  -  1.7 

n/a 

<  100  k 

P-2 

1 

Pacific  Sierra 
Research 

SRS 

n/a 

0.35  -  1.1 

n/a 

<60 

P-2 

1 

ABB  Bomem 

MR254 

n/a 

Dual  bands 

n/a 

10-82 

P-3 

1 

ABB  Bomem 

MR255 

n/a 

Dual  bands 

n/a 

10-82 

P-3 

1 

P-3 

High  Speed  UV 
Spectrometer 

n/a 

0.01  -0.4 

0.25  -0.29 

<500 

P-3 

1 

PIMAX 

1340  Pixis  Visible 
Spectrometer 

n/a 

0.4  -  0.77 

0.3-  1.0 

<  183 

P-3 

1 

PIMAX 

1024  Pixis  UV-Vis 
Spectrometer 

n/a 

0.01  -0.4 

0.2  -  0.9 

<  183 

P-3 

1 

Zeiss 

Zeiss  Spectrometer 

n/a 

Dual  bands 

0.2-  1.0 

<95 

P-3 

1 

Cl  Systems 

Cl  Systems 

n/a 

Other 

n/a 

Varies 

P-4 

1 

Bomem 

n/a 

n/a 

Other 

n/a 

n/a 

P-7 

2 

Cary 

5E  UV-VIS-NIR 
Spectrometer 

n/a 

0.18-2.5 

n/a 

n/a 

P-B 

1 

Thermal 

Electron 

Nicolet  Magna 

n/a 

2.5  -  25 

n/a 

n/a 

P-B 

1 
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TABLE  5-30.  OTHER  PARAMETERS  AND  INFORMATION  FOR  THE  SPECTRAL 

RADIOMETERS 

Supplier 

Model 

Lenses  /  FOV 

Resolution 

(cm1) 

Year 

Purchased 

Software 

Calibration 

Sources 

Comments 

Owner 

\BB 

Bomem 

FTIR  MR254 

n/a 

n/a 

n/a 

n/a 

n/a 

n/a 

P-1 

21 

Systems 

SR5000 

n/a 

n/a 

n/a 

n/a 

n/a 

n/a 

P-1 

\BB 

Bomem 

MR304SC 

3 

n/a 

2004 

W  ritten-in-house 

BB,  spectral 
lamps 

n/a 

P-2 

SciTec 

8145 

None 

n/a 

2003 

Custom  built  in 
C++ 

Done  at 
SciTec 

n/a 

P-2 

Pacific 

Sierra 

Research 

SRS 

14mm,  19mm, 
72mm,  238mm, 
540mm 

n/a 

1996 

(upgraded 

2006) 

Custom  built  in 
LabVIEW 

Spectral  lamps, 
common  sources 

n/a 

P-2 

\BB 

Bomem 

MR254 

0.1,  1.0,  2.5  deg 

1,2,  4,  8,  16 

2000 

Bomem  & 
Custom 

BB,  spectral 
lamps 

2  ch,  3  det  options  (Ge, 
InSb,  HgCdTe) 

P-3 

\BB 

Bomem 

MR255 

0.1,  1.0,  2.5  deg 

1,2,  4,  8,  16 

2001 

n/a 

BB,  spectral 
lamps 

2  ch,  3  det  options  (Ge, 
InSb,  HgCdTe) 

P-3 

P-3 

High  Speed 

UV 

Spectrometer 

Variable  optics, 
2.5  deg 

1  nm 

2000 

HW  specific 

Spectral  lamps 

Proxitronic  ASB 
intensifier  1  x  128 

P-3 

P1MAX 

1340  Pixis 

Visible 

Spectrometer 

Variable  optics, 
0.25  to  5  deg 

0.11-2.5 

nm 

2006 

HW  specific 

Spectral  lamps 

Camera:  Pixis  1340  x 
100 

P-3 

P1MAX 

1024  Pixis 

UV-Vis 

Spectrometer 

Variable  optics, 
0.25  to  5  deg 

0.11-2.5 

nm 

2006 

HW  specific 

Spectral  lamps 

Camera:  Pixis  1024  x 
256  UV  enhanced 
intensified 

P-3 

Zeiss 

Zeiss 

Spectrometer 

0.3  deg 

2.3  nm 

1999 

HW  specific 

Spectral  lamps 

Zeiss  Spectrometer 

1  x  1024 

P-3 

21 

Systems 

Cl  Systems 

n/a 

0.4-14  pm 

1998 

Custom 

BB,  Other 

0.4  -  14  micrometers 

P-4 

Bomem 

n/a 

Varied 

n/a 

2004 

Rtools/ Acquire/In 
-house 

BB 

n/a 

P-7 

2ary 

5E  UV-VIS- 
NIR 

Spectrometer 

n/a 

n/a 

1993 

ADL  (Varian 
proprietary) 

Reference 

coupons, 

vender 

Commercial  system 
with  DHR,  BRDF, 
BTDF  attachments; 
DHR  sample  port  size 
1",  1.25" 

P-B 

Thermal 

Electron 

Nicolet  Magna 

n/a 

n/a 

1994 

Omnic 

Reference 

coupons, 

vender 

Commercial  system 
with  DHR,  BRDF, 
BTDF  attachments; 
DHR  sample  port  size 
0.875" 

P-B 

f.  Reflectometer.  Reflectometers  with  major  characteristics  and  their  owners  are  listed 
in  Table  5-31.  Other  parameters  and  information  are  listed  in  Table  5-32.  Comments 
on  the  reflectometers  are  listed  in  Table  5-33. 
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TABLE  5-31.  REFLECTOMETERS  AND  SPECIFICATIONS 

Supplier 

Model 

Spectral  Range  (pm) 

Sample  Spot  Size 

Owner 

Qty 

(Unit) 

Surface  Optics 

SOC-400T 

2-25 

n/a 

P-1 

1 

Analytical 

Spectral 

Devices 

Fieldspec-FR 

0.3  -  2.5 

n/a 

P-1 

1 

P-B 

Imaging  Sphere 

0.4  -  12 

0.375";  f/23 

P-B 

1 

P-B 

Imaging  Sphere 
Environmental 

0.4  -  12 

0.375";  f/23 

P-B 

1 

P-B 

Portable  Reflectometer 

1 

0.4  -  12 

0.375";  f/7.2 

P-B 

1 

P-B 

Portable  Reflectometer 

2 

0.4  -  12 

0.375";  f/7.2 

P-B 

1 

P-B 

IPBR  1 

0.4  -  12 

0.125";  f/20 

P-B 

1 

P-B 

IPBR2 

0.4  -  12 

0.125"  to  1";  f/20 

P-B 

1 

P-B 

OPBR  1 

0.4  -  12 

0.25"  to  1";  f/25 

P-B 

1 

TABLE  5-32.  OTHER  PARAMETERS  AND  INFORMATION  FOR  THE  REFLECTOMETER 

Supplier 

Model 

Year 

Purchased/ 

Developed 

Software  Used 

Calibrated 

with 

Owner 

Surface  Optics 

SOC-400T 

n/a 

n/a 

n/a 

P-1 

Analytical 
Spectral  Devices 

Fieldspec-FR 

n/a 

n/a 

n/a 

P-1 

P-B 

Imaging  Sphere 

1986  -  1993 

In-house  propriety, 
ORACLE,  OFFICE 

Reference 

coupons 

P-B 

P-B 

Imaging  Sphere 
Environmental 

1993  -  1996 

In-house  propriety, 
ORACLE,  OFFICE 

Reference 

coupons 

P-B 

P-B 

Portable  Reflectometer  1 

1989  -  1991 

In-house  propriety, 
ORACLE,  OFFICE 

Reference 

coupons 

P-B 

P-B 

Portable  Reflectometer  2 

1998  -  2000 

In-house  propriety, 
ORACLE,  OFFICE 

Reference 

coupons 

P-B 

P-B 

IPBR  1 

1986  -  1991 

In-house  propriety, 
ORACLE,  OFFICE 

Reference 

coupons 

P-B 

P-B 

IPBR  2 

1992  -  1993 

In-house  propriety, 
ORACLE,  OFFICE 

Reference 

coupons 

P-B 

P-B 

OPBR  1 

1996  -  2000 

In-house  propriety, 
ORACLE,  OFFICE 

Reference 

coupons 

P-B 
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TABLE  5-33.  COMMENTS  ON  THE  REFLECTOMETERS 

Supplier 

Model 

Comments 

Owner 

Surface  Optics 

SOC-400T 

n/a 

P-1 

Analytical 
Spectral  Devices 

Fieldspec-FR 

n/a 

P-1 

P-B 

Imaging  Sphere 

Ambient  samples,  high  LWIR  precision  (0.064%);  DHR 

P-B 

P-B 

Imaging  Sphere 
Environmental 

Heated  samples  (up  to  700F);  DHR 

P-B 

P-B 

Portable  Reflectometer  1 

In-situ  samples,  vehicle  surfaces;  DHR 

P-B 

P-B 

Portable  Reflectometer  2 

In-situ  samples,  vehicle  surfaces;  DHR 

P-B 

P-B 

IPBR  1 

Ambient  samples,  grazing  incident/viewing,  monostatic 
geometry,  in-plane  bidirectional 

P-B 

P-B 

IPBR2 

Ambient  samples,  grazing  incident/viewing,  monostatic 
geometry,  in-plane  bidirectional 

P-B 

P-B 

OPBR  1 

Ambient  samples,  grazing  incident/viewing,  full 
hemispheric  geometry,  out-of-plane  bidirectional 

P-B 

f.  Other  Types.  Other  types  of  instruments  are  listed  in  Table  5-34. 


TABLE  5-34.  OTHER  TYPES  OF  INSTRUMENTS 


Supplier 

Radiometer 

Model 

Spectral 
Range  (pm) 

Frame 
Rate  (Hz) 

Software 

Year 

Purchased 

Calibrated 

with 

Comments 

Owner 

Qty 

(Unit) 

- 

Custom 

0.77  -  1.2 

Varies 

Custom 

2004 

Other 

sources 

NIR  Suite 
containing 
Laboratory  and 
Tactical  systems 

P-4 

1 

TI 

TI 

Other 

Varies 

Custom 

2004 

None  needed 

Tactical  unit 

P-4 

1 
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CHAPTER  6 

DRAFT  DOCUMENT  STANDARDS  FOR  CHARACTERIZING  THE  MEASUREMENT 
CAPABILITIES  OF  EO/IR  MEASUREMENT  SYSTEMS 

This  chapter  is  fundamentally  based  on  a  document  entitled.  Draft  of  Electro-Optical 
Standard  for  Applications,  delivered  10  October  2006  to  the  SMSG-18  task  lead,  Mr.  Jeffrey  W. 
Burks  (EO/IR  Laboratory,  Code  SNS  (now  Code  RYS)  at  the  AFRL  at  Wright-Patterson  AFB. 
The  document  was  prepared  under  contract  by  EO/IR  Remote  Sensing  Group,  Mission  Research 
and  Technical  Service  Division,  Alliant  Techsystems  Inc. 

6.1  Executive  Summary 

6.1.1  Background.  As  part  of  the  effort  to  serve  the  technical  and  operational  needs  of  member 
test  ranges  and  facilities,  the  RCC  established  the  SMSG  to  develop  signature  measurement 
standards  covering  the  electromagnetic  spectrum.  The  SMSG  focused  on  specific  areas  of  radar, 
millimeter  wave,  infrared,  laser,  visible,  ultraviolet,  seismic,  acoustic,  and  magnetic. 

6.1.2  Scope.  This  draft  documents  standards  for  characterizing  measurement  capabilities  of 
EO/IR  measurement  systems  and  helps  establish  a  demonstration  program  that  verifies  and 
quantifies  the  improvement  in  measurement  capability  that  will  result  from  implementing 
documented  standards.  The  standard  will  define  levels  of  information  contents  that  are 
synonymous  with  levels  of  knowledge  and  the  requirements  to  meet  these  levels.  For  example, 
the  lowest  level  would  be  capable  of  providing  qualitative  information.  A  middle  level  might 
provide  relative  quantitative  information.  The  highest  level  incorporates  both  lower  levels  and 
could  provide  quantitative  information  with  specified  measurement  uncertainties.  The  allowed 
levels  are  intended  to  specify  cost-effective  options  to  meet  preset  test  requirements.  The 
standard  includes: 

a.  EO  instrument  characterization  and  calibration  procedures. 

b.  Standard  data  reduction  procedures  to  transform  raw  data  to  engineering  units. 

c.  Measurement  uncertainty  analysis. 

d.  Standard  terminology  and  units  of  measure. 

This  draft  provides  standards  for  planning,  executing,  and  reporting  signature 
measurements.  It  also  describes  the  resources  available  to  the  IR  range  community  and 
recommends  procedures  to  help  testers  make  efficient,  effective  use  of  instrumentation. 
Compliance  with  the  standard  and  use  of  the  recommended  procedures  will  result  in  well- 
documented  calibration  and  test  data  sets  available  to  all  users  within  the  SMSG. 

Following  completion  of  the  final  product,  new  task  proposals  will  be  prepared  at  two, 
three,  or  four-year  intervals  to  revise  and  update  the  standard.  Conceivably,  some  of  the  standard 
will  require  frequent  updates  while  others  will  require  less  frequent  updates. 

As  part  of  the  deliverables  of  the  first  year  (and  second  year,  if  funded),  the  Updated 
Objectives  will  include  a  schedule  for  updates  based  on  assessments  of  the  applicable 
technologies  at  that  time. 
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6.2  Overview 

The  following  definition  will  be  used  throughout  this  document.  Electro-optical  (EO) 
refers  to  sensors  and  associated  measurement  systems  used  to  characterize  infrared  through 
ultraviolet  (25  micrometers  to  0.2  micrometers)  radiation.  Measurement  systems  include  optical 
components  and  photon-sensitive  electronic  digital  detectors. 

In  this  document,  the  EO  standard  for  application  is  organized  in  three  levels  of 
knowledge  with  each  level  covering  three  to  four  categories: 

a.  Level  I.  Basic  measurement  capability  for  qualitative  detection  or  bookkeeping. 

b.  Level  II.  Relative  identification  of  general  target  characteristics  For  example, 

relative  radiance  or  products  derived  from  relative  radiance  such  as 
relative  spatial  reflectivity  and  relative  spatial  temperature  map. 

c.  Level  III.  Absolute  measurement  on  specific  target  properties. 

Common  descriptors  for  all  three  levels  include  applicable  techniques,  instrumentation 
involved,  experimental  setup,  operator  qualification,  cost  factor,  data  acquisition,  data  analysis, 
and  data  report.  Potential  characteristics  for  all  the  levels  are  listed  in  Table  6-1. 


TABLE  6-1.  DESCRIPTORS  VS  LEVELS  1 

No. 

Descriptors 

Level  I 

Level  II 

Level  III 

1 

Spectral  Band  Choice 

Important 

Important 

Important 

2 

Field  of  View  (FOV) 

Important 

Important 

Important 

3 

Detector  Dynamic  Range 

Important 

Important 

Important 

4 

ADC  requirement 

low 

medium 

important 

5 

Signal  Conditioning 

Yes 

Yes 

Yes 

6 

Background  correction 

None 

Some 

Important 

7 

Data  Acquisition 

Yes 

Yes 

Yes 

8 

Data  Processing 

Minimum 

Some 

Important 

9 

Data  Analysis 

Yes 

Yes 

Yes 

10 

Data  Reporting 

Yes 

Yes 

Yes 

11 

Detector  System  Size  and  Weight 

Consideration 

Consideration 

Consideration 

12 

DC  Powered  Option 

Consideration 

Consideration 

Consideration 

13 

Cable  Harness 

Consideration 

Consideration 

Consideration 

14 

Environment  Issues 

Yes 

Yes 

Important 

15 

Relative  Calibration  (on-the-flight) 

No 

Yes 

Yes 

16 

Absolute  Calibration  (traceable) 

No 

No 

Yes 

17 

Uncertainty  Analysis 

No 

Yes 

Yes 

18 

Image  Quality 

low 

medium 

high 

19 

Field  Test  Potential 

Yes 

Yes 

No 

20 

Setup  or  alignment  requirement 

Low 

Medium 

Higher 

21 

Operator  Qualification 

Intern  Level 

Junior 

Engineer 

Senior  Scientist 

22 

Overall  Cost  Factor 

Low 

Medium 

Higher 
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6.2. 1  Spectral  Band.  It  is  essential  to  select  appropriate  spectral  band  for  a  given  application  to 
optimize  detection  sensitivity,  dynamic  range,  signal  noise  ratio,  etc.  However,  situations  do 
arise  in  which  one  needs  to  detect  signals  in  a  predefined  spectral  region  where  detection 
condition  is  not  optimized  (due  to  either  the  nature  of  the  signal  or  the  limitation  imposed  upon 
by  instrumentation  availability).  In  any  case,  the  trade-off  of  spectral  band  selection  is: 

a.  Spectral  response 

b.  S/N  ratio 

c.  Detector  dynamic  range 

d.  Image  contrast 

e.  Cost 

It  is  also  critically  important  in  the  selection  of  windows,  lenses,  and  detector  after  the 
spectral  band  and  bandwidth  are  determined. 

6.2.2  Field  of  View  (FOV).  For  IR  signature  measurements,  FOV  requirement  is  decided  by 
target  size  and  the  distance  from  the  target  to  the  sensor.  On  the  other  hand,  the  system  FOV 
depends  on  the  instrument  configuration.  Thus,  it  is  important  to  estimate  the  required  FOV  for 
signal  detection  based  on  each  specific  application  and  then  configure  a  sensor  system  that  meets 
the  FOV  requirement.  In  case  of  IR  imaging  application,  this  involves  selection  of  a  camera  lens 
that  has  appropriate  focal  length  and  f-number.  The  impact  of  instrument  FOV  is: 

a.  S/N  ratio. 

b.  Spatial  resolution  (scanning). 

c.  Temporal  or  spatial  dynamic  range  (moving  target). 

d.  Cost. 

6.2.3  Dynamic  Range.  Detector  dynamic  range  will  affect  the  following: 

a.  S/N  ratio. 

b.  Image  contrast. 

c.  Measurement  accuracy. 

6.2.4  ADC  Requirement.  ADC  with  better  digital  resolution  is  required  for  Level  III 
knowledge  in  order  to  fully  utilizing  the  potential  of  a  higher  end  camera.  The  impact  is: 

a.  Accuracy. 

b.  Resolution. 

c.  Cost. 

6.2.5  Signal  Conditioning.  Depending  on  specific  application,  the  signal  conditioning  required 
by  radiometric  sensors  can  be  very  diverse.  It  is  important  that  the  appropriate  signal 
conditioning  be  used  to  fully  realize  the  potential  of  a  given  sensor  system. 

For  example,  one  of  the  important  parameters  associated  with  detector/preamplifier 
combination  is  the  electric  bandwidth  which  is  defined  as  the  range  of  frequencies  within  which 
the  amplifier  will  respond.  This  frequency  range  is  often  measured  between  the  half  power 
(3-dB)  points  on  the  output  response  versus  the  frequency  curve  for  a  constant  input.  For  various 
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reasons,  the  bandwidth  of  the  signal  electronics  is  often  tailored  by  using  electronic  filters.  For 
example,  high  and  low  pass  electronic  filters  are  often  applied  to  remove  the  60  Hz  noise  from 
the  power  grid  and  noise  of  frequencies  outside  the  frequency  band  of  interest. 

It  is  also  important  that  the  gain  and  noise  figure  of  the  preamplifier  be  adequate  to  assure 
that  the  S/N  ratio  of  the  detector  is  not  seriously  degraded  by  the  remainder  of  the  system.  The 
noise  figure  is  defined  as  the  ratio  between  the  S/N  of  the  input  to  an  amplifier  to  the  S/N  at  the 
output.  Noise  figures  of  near  unity  are  desirable  in  the  detector  and  amplifier,  as  are  high 
sensitivities  or  gains  in  the  first  stage  of  a  circuit.  Noise  becomes  progressively  less  critical  in 
amplifier  stages  with  gains  greater  than  unity  added  beyond  the  preamplifier. 

Other  common  signal  conditioning  procedure  includes  the  use  of  synchronous 
demodulation  or  phase  sensitive  detection  (a  type  of  correlation  analysis).  This  procedure 
involves  the  use  of  a  chopper-modulated  signal  with  a  lock-in  amplifier  to  remove  DC  offset 
from  the  signal  and  to  improve  the  S/N  ratio  of  the  sensor  system. 

6.2.6  Background  Correction.  Background  correction  is  essential  for  Level  III  knowledge  and 
includes  the  following: 

a.  Background  object  and  atmosphere  emission. 

b.  Background  object  and  atmosphere  transmission. 

c.  Background  object  and  atmosphere  reflection. 

6.2.7  Data  Acquisition.  Data  acquisition  (DAQ)  is  the  act  of  recording  the  signal  in  a  form  that 
is  amenable  to  analysis.  Common  forms  of  DAQ  are: 

a.  Voltage  or  current  taken  from  various  sensors  during  the  course  of  an  experiment. 

b.  Analog/Digital  imageries  taken  with  cameras. 

c.  X-Y  plotters. 

d.  Microdensitometers. 

e.  Strip  charts. 

f.  Analog  tape  recorders. 

g.  Digital  data  acquisition  has  become  the  standard  and  preferred  form.  Typical  digital 
DAQ  system  contains  a  multiplexer,  a  sample  and  hold  amplifier,  an  analog  to  digital 
converter  (ADC),  an  interface  to  a  central  processing  unit,  high  speed  and  bulk 
memory,  and  input/output  (I/O)  devices. 

The  important  considerations  for  the  multiplexer  are  its  programmability,  range  of 
permissible  voltages  or  dynamic  range,  and  switching  speed  or  multiplexer  settling  time;  for  the 
sample  and  hold  amplifier,  the  acquisition  and  operative  times  along  with  their  settling  times. 

The  acquisition  time  is  the  delay  between  the  time  the  hold  control  signal  is  applied  and  the 
actual  time  the  circuit  enters  the  hold  mode.  The  ADC  may  take  many  forms.  Popular  types  are 
the  integration,  successive  approximation,  tracking,  multi  comparator  ladder  and  voltage  to 
frequency  converters.  The  questions  arising  when  applying  ADC’s  are  usually  how  rapidly  and 
accurately  are  the  conversions  made.  Information  theory  (by  Shannon  or  Nyquist)  suggests  that 
the  conversion  should  occur  at  a  rate  at  least  twice  the  maximum  frequency  of  interest. 


6-4 


RCC  Special  Report:  Development  Of  Electro-Optical  (EO)  Standard  Processes  for  Application,  November  2011 


6.2.8  Data  Processing.  The  signal  recorded  by  the  data  acquisition  system  is  retrieved  and 
processed  in  some  fashion  to  yield  the  final  information  desired  from  the  measurement  system. 

In  general,  data  processing  is  synonymous  with  the  digital  DAQ  process  followed  by  computer 
aided  data  manipulation  and  graphic  presentation  of  results.  The  data  processing  step  might  be 
as  simple  as  the  addition  of  units  of  measure  on  a  strip  chart  or  as  complex  as  the  digitization, 
scaling,  and  Fourier  transformation,  etc.  to  extract  desired  information.  Data  processing  includes 
viewing  the  data  as  it  is  processed  and  archiving  the  data.  Usually,  data  processing  is  minimum 
for  Level  I  applications;  some  data  processing  is  needed  for  Level  II  applications;  extensive  data 
processing  is  necessary  for  Level  III  applications. 

Typical  data  processing  steps  include: 

a.  Conditioning  of  the  analog  signal  to  provide  appropriate  levels  and  bandwidths  of 
signals  for  the  digitizer. 

b.  Actual  digitization  of  the  data  and  the  transfer  to  computer  memory. 

c.  Storage  of  the  data  on  bulk  memory  for  later  processing  or  reprocessing  if  required. 

d.  Retrieval  and  reformatting  of  the  data  for  subsequent  file  manipulations. 

6.2.9  Data  Analysis.  Data  analysis  is  generally  performed  after  the  measurements.  Immediate 
minimal  data  analysis  should  be  accomplished  along  with  the  generation  of  data  to  establish  error 
bonds  and  to  identify  anomalies  with  their  causes,  such  as  the  effects  due  to  known  environment 
perturbations,  instrument  effects,  etc. 

6.2.10  Data  Reporting. 

a.  Reporting  contents  for  Level  I  knowledge  are: 

(1)  Image. 

(2)  Date  and  time  stamps. 

(3)  Test  notes  (setup,  instruments,  environment). 

b.  For  Level  II  knowledge,  the  contents  include: 

(1)  Images. 

(2)  Numerical  values. 

(3)  Graphs. 

(4)  Theoretic  model  used. 

(5)  Measurement  equation. 

(6)  Test  notes  (setup,  instruments,  environment). 

c.  For  Level  III  knowledge,  the  content  reported  includes  the  above  for  Level  II 
knowledge  as  well  as: 

(1)  Calibration  procedure  and  method. 

(2)  Calibrations  standard  used. 

(3)  Uncertainty  analysis. 

(4)  References  used. 

6.2. 1 1  Detector  Size  and  Weight.  Important  issue  for  field  test  applications. 
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6.2.12  DC  Power  Option.  A  consideration  for  all  levels,  depends  on  specific  situation 

6.2.13  Cable  Harness.  The  cable  harness  is  important  for  the  following  reasons: 

a.  Potential  noise  generation  due  to  contact  problem. 

b.  Minimize  noise  pickup  by  using  well-shielded  cables. 

c.  Minimize  cable  tension  at  detector  to  improve  setup  stability. 

6.2.14  Environmental  Issues.  Environmental  condition  is  vital  to  Level  III  knowledge  since  it 
will  affect  measurement  accuracy  and  reproducibility. 

6.2.15  Relative  Calibration.  Relative  calibration  provides  a  relative  scaling  factor  to 
measurement  result.  Depending  on  the  specific  application,  the  calibration  could  be  for  sensor 
spectral,  spatial,  and  temporal  response  with  known  sources;  its  linearity  or  nonlinearity  for 
possible  inputs  must  be  determined;  parameter  such  as  FOV,  frequency  response,  and  the  like 
must  be  known.  For  IR  measurements,  the  calibration  can  be  performed  in  any  lab  or  on  the 
flight  during  field  tests  against  a  thermally  stable  source  with  known  spectral  contents.  In 
general,  this  type  of  calibration  is  also  traceable  to  an  absolute  calibration.  In  this  case,  the 
output  signal  from  a  relatively  calibrated  sensor  can  be  directly  related  to  a  transfer  standard. 

For  example,  calibration  of  a  radiometer  system  with  a  linear  response  to  input  radiation 
requires  a  number  of  steps: 

a.  Determine  calibration  technique  for  in-band  radiance,  radiant  intensity,  and 
irradiance. 

b.  Determine  instrument  spectral  characteristics. 

c.  Use  known  source  to  generate  instrument  output  (voltage  or  current)  signal. 

d.  Determine  source  output  (for  example,  generate  Planckian  curves  and  geometry). 

e.  Repeat  calibration  steps  at  least  twice. 

f.  Calculate  the  instrument  response  (for  example,  for  a  radiance  calibration). 

6.2.16  Absolute  Calibration.  This  type  of  calibration  is  traceable  to  NIST  standard  and  should 
be  compliant  with  international  standard.  It  provides  absolute  scaling  to  the  measured  result  and 
need  to  be  performed  periodically  in  dedicated  facilities.  The  standard  calibrated  against  is 
accompanied  with  uncertainty  analysis  as  well  as  signed  and  dated  calibration  certificate. 

First,  the  calibration  requirement  is  determined  from  the  intended  instrument  application. 
Then,  a  suitable  calibration  source  is  positioned  to  either  overfill  or  under-fill  the  FOV  of  the 
instrument,  depending  upon  the  type  of  calibration  required.  For  example,  absolute  IR  camera, 
IR  radiometers,  or  IR  spectrometer  calibration  involves  measuring  the  emissivity  of  a  blackbody 
cavity  as  a  function  of  temperature,  wavelength,  distance,  aperture  size,  FOV,  etc.  to  determine 
the  response  of  the  instrument.  In  these  IR  calibrations,  the  known  sources  are  Planckian 
radiators  which  behave  nearly  like  blackbodies.  The  calibration  source  radiance  is  calculable, 
within  an  error  band  related  to  the  accuracy  of  their  measured  temperature,  emissivity,  and 
uniformity,  using  the  well-known  Planck  equation.  In  the  case  of  a  radiance  calibration  the 
source  is  situated  nearby  (possibly  with  an  evacuated  path  to  the  sensor)  to  overfill  the  FOV  of 
the  instrument.  The  distance  from  the  source  to  the  instrument  is  not  needed  for  further 
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calculations.  For  radiant  intensity  calibrations,  the  source  is  situated  to  under-fill  the  instrument 
FOV.  In  this  type  of  calibration,  the  distance  from  the  instrument  to  the  source  and  the  area  of 
the  source  are  required  for  subsequent  calculations.  Typically  several  sources,  with  varying 
temperatures,  ranges,  areas,  FOVs,  etc.,  are  viewed  and  at  least  two  measurements  are  required. 

In  general,  signals  are  recorded  with  sources  at  two  or  more  different  known  output 
conditions.  A  multiplicative  instrument  response  is  derived  from  these  measurements,  the  output 
of  the  source  for  the  two  different  conditions  are  determined  from  measurements  (with  standard 
detectors,  manuals,  calculations,  etc.)  and  the  difference  is  calculated.  The  difference  between 
the  signals  recorded  when  exposed  to  the  sources  under  two  different  conditions  is  then 
determined.  The  instrument  response  is  the  ratio  of  the  two  differences.  For  example,  in  one 
type  of  calibration,  the  instrument  signal  might  be  measured  in  volts  and  the  source  radiant 
output  in  watts  per  steradian  within  a  specified  spectral  band.  The  multiplicative  response  would 
then  be  a  number  with  units  of  watts  per  steradian  per  volt  within  the  specified  bandpass  which 
could  be  used  as  a  multiplier  of  voltages  measure  while  viewing  unknown  sources  to  yield  the 
inband  radiant  intensity  attributable  to  the  source. 

Typical  IR  camera,  radiometer,  or  spectrometer  calibration  includes  the  following  steps: 

a.  Calculate  radiances  of  the  calibration  sources  using  at  least  two  different 
temperatures. 

b.  Find  the  transmission  and  reflection  of  the  filters,  windows  (optical  or  atmospherical), 
lenses,  mirrors  etc. 

c.  Find  the  spectral  response  of  the  camera  or  radiometer. 

d.  Find  the  normalized  product  of  the  spectral  transmission  of  the  filters,  windows, 
lenses,  mirrors,  detectors,  etc. 

e.  Find  the  product  of  the  source  radiance  and  normalized  transmission  term. 

f.  Integrate  the  calculated  spectral  radiance/transmission  product  and  measure  the 
corresponding  camera/radiometer  voltage  (Atmospheric  effect  may  also  be  treated 
within  this  step). 

g.  Calculate  the  instrument  response  (i.e.,  the  slope  of  the  radiance  vs.  voltage  function). 

h.  The  radiance  of  the  test  target  is  then  the  product  of  the  instrument  response  and  the 
voltage  measured  while  viewing  the  target. 

6.2.17  Uncertainty  Analysis.  Uncertainty  analysis  is  required  for  Level  II  and  Level  III 
applications.  The  methodology  and  arithmetic  of  uncertainty  analysis  is  presented  in  numerous 
references  and  will  not  be  discussed  in  this  draft.  The  key  to  perform  uncertainty  analysis  for 
radiometric  instruments  is  to  correctly  identify  and  estimate  the  magnitude  of  error  associated 
with  each  error  source.  The  errors  must  also  be  classified  as  bias  or  precision.  The  elemental 
errors  have  to  be  summed  up  with  appropriate  mathematical  scheme  to  determine  the  final 
uncertainty  limit.  Possible  sources  of  error  associated  with  radiometric  instrumentation  are: 

a.  The  parameters  related  to  the  calibration  source  such  as  temperature,  emissivity,  and 
uniformity. 

b.  Geometry  parameters  such  as  the  source  to  sensor  distance,  source  area,  instrument 
FOV. 

c.  The  atmospheric  transmission,  reflection,  and  absorption. 
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d.  The  spectral  characteristics  of  the  detectors,  filters,  lenses,  and  windows. 

e.  The  gain,  bias,  drift,  linearity,  frequency  response,  sensitivity,  dynamic  range,  spatial 
and  spectral  resolution  of  the  instrument  and  associated  data  acquisition  system. 

6.2.18  Image  Quality.  The  purpose  of  measuring  the  quality  of  an  image  is  to  decide  the 
usefulness  of  an  image  to  a  specific  application  and  to  evaluate  the  design  or  operation  of  an 
image  system.  The  National  Imagery  Interpretability  Rating  Scale  (NIIRS)  defines  and  measures 
the  quality  of  images  and  performance  of  an  imaging  system.  The  NIIRS  criteria  are  based  on 
textual  descriptors  to  define  the  interpretability  of  an  image  at  a  specific  NIIRS  level.  There  are 
total  of  55  NIIRS  criteria  for  10  levels.  Six  criteria  are  for  each  level  from  levels  1  through  9.  A 
single  criterion  is  at  level  0  (the  lowest  level).  Based  on  this  scale,  an  image  can  be  evaluated 
through  a  rating  process.  As  the  result,  the  given  image  is  assigned  a  number  (from  0  to  9)  to 
indicate  its  interpretability.  The  higher  the  assigned  number,  the  higher  the  imagery 
interpretability  is.  Because  the  NIIRS  criteria  are  strongly  based  on  spatial  information,  more 
information  can  be  extracted  from  an  image  with  a  higher  NIIRS  rating.  Thus,  the  NIIRS 
provides  a  systematic  approach  to  measuring  the  quality  of  photographic  or  digital  imagery,  the 
performance  of  image  capture  devices,  and  the  effects  of  image  processing  algorithms. 

By  referring  to  the  NIIRS  criteria,  an  example  of  progression  in  spatial  detail  for  the 
application  of  target  characterization  is  given  in  paragraph  6.9. 

6.2.19  Field  Test  Potential.  Level  I  and  Level  II  measurements  are  applicable  to  field  test  or 
general  laboratory  conditions.  Level  III  knowledge  is  usually  meant  for  laboratory  with 
environment  controls  for  accurate  and  reproducible  measurement.  Depending  on  the  specific 
application,  Level  III  knowledge  might  also  be  pursued  in  open  field.  For  example,  if  you  are 
interested  in  the  IR  signature  of  a  new  boat  on  a  calm  open  ocean  at  night  it  would  be  possible  to 
measure  with  absolute  calibrated  and  spatially  corrected  radiances  that  would  qualify  as  a  Level 
III  measurement. 

6.2.20  Setup  or  Alignment  Requirement.  Level  I  setup  or  alignment  requires  minimal  effort; 
Level  II  fidelity  require  substantial  effort  in  test  setup  and  instrument  alignment;  Level  III 
fidelity  demands  stringent  setup  or  alignment  conditions  in  order  to  reproduce  accurate  result. 

6.2.21  Operator  Qualification.  It  is  assumed  that  the  experiment  or  test  setup  is  already 
finalized.  Different  levels  of  skills  or  experience  are  expected  from  an  operator  to  collect  data, 
processing  data,  and  report  data  for  different  level  of  applications.  In  general,  senior  level 
scientists  are  always  involved  in  the  designing  of  the  experiment  setup  to  specific  applications 
and  in  validating  as  well  as  in  interpreting  the  measurement  results. 

6.2.22  Overall  Cost  Factor.  Overall  cost  factor  depends  on  specific  aspects  such  as: 

a.  Level  of  fidelity. 

b.  Experiment  complexity. 

c.  Experimental  conditions. 

d.  Instrumentation  requirement. 

e.  Operator  education  background  and  level  of  experience. 
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6.3  Level  I  Fidelity 

6.3.1  Level  I  Fidelity  Situations.  Level  I  applies  to  the  following  situations: 

a.  For  applications  where  qualitative  results  are  sufficient. 

b.  For  record  keeping  and  photo  documentary  purposes. 

c.  For  labs  or  sites  where  only  basic  instrumentations  are  available. 

6.3.2  Level  I  Fidelity  Information.  Information  for  Level  I  includes  but  is  not  limited  to: 

a.  Relative  temperature  distribution  (contrast). 

b.  Relative  geometry  (shape  and  size). 

c.  Relative  brightness  (contrast). 

d.  Target  material  type  (metal  or  non-metal). 

e.  Relative  motion  (against  background). 

f.  Transient  effect  (flashes). 

6.4  Level  II  Fidelity 

6.4. 1  Level  II  Fidelity  Situations  Level  II  fidelity  is  a  mid  step  between  Level  I  and 
Level  III.  Certain  overlaps  between  the  levels  are  expected,  and  Level  II  mainly  applies  to 
situations  where  qualitative  results  are  not  sufficient  and  approximate  numerical  values  are 
needed.  In  Level  II,  relative  calibration  on  instrument  is  required. 

6.4.2  Level  II  Fidelity  Information.  Information  sought  for  Level  II  fidelity  might  include  the 
following: 


a.  Characteristic  temperature  (average,  maximum,  minimum). 

b.  Thermal  conductivity  (typical  values). 

c.  Radiance  (typical  values). 

d.  Transmittance  (typical  values). 

e.  Reflectance  (typical  values). 

f.  Emissivity  (typical  values). 

g.  Surface  roughness  (typical  values). 

h.  Polarization  state  (dominant  component). 

i.  Spatial  frequency  (textures  or  patterns). 

j.  Physical  properties  (size  and  contour). 

6.5  Level  III  Fidelity 

6.5.1  Level  III  Fidelity  Situations.  Level  III  fidelity  applies  to  applications  where  accurate, 
reliable,  and  reproducible  numerical  results  are  required.  Absolute  and  traceable  instrument 
calibrations  are  required  for  this  level  of  fidelity. 
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6.5.2  Level  II  Fidelity  Information.  Information  sought  under  Level  III  fidelity  includes  those 
sought  under  Level  I  and  Level  II  fidelities  as  well  as  the  following: 

a.  Absolute  temperature  distribution  and  gradient. 

b.  Absolute  radiance. 

c.  Absolute  emissivity. 

d.  Absolute  reflectivity. 

e.  Absolute  surface  roughness. 

f.  Degree  of  polarization. 

g.  Field  phase  distribution. 

6.6  Examples:  Level  I 

6.6.1  Printed  Circuit  Board.  Infrared  thermal  imaging  can  be  used  to  measure  steady  state 
temperature  distribution  on  a  printed  circuit  board  for  trouble  shooting  or  circuit  diagnostics. 

a.  Close  up  lens  with  larger  FOV. 

b.  Larger  thermal  dynamic  range. 

c.  Manual  NUC  correction. 

d.  Real-time  monochrome  video  signal  output  from  camera. 

e.  Real-time  image  displaying. 

f.  Image  storage  medium. 

6.6.2  Fire  in  Burning  Building.  LWIR  or  MWIR  spectral  band  could  be  the  choice.  This 
application  might  require  an  IR  camera  with  faster  time  constant  in  order  to  capture  transient 
thermal  effects  such  as  flame  with  changing  intensity  and  direction,  etc.  More  often,  an  absolute 
surface  temperature  is  desired  in  this  situation  to  assess  whether  a  victim  should  be  rescued 
immediately,  whether  a  door  should  be  open,  etc. 

6.6.3  Deer  on  Road.  LWIR  (7-14  pm)  camera  would  be  the  first  choice  for  earlier  detection  of 
road  hazards  for  vehicles  traveling  during  night.  The  camera  should  increase  a  driver’s  night 
visibility  three  to  five  times  the  normal  range  of  vehicle  headlights.  In  the  LWIR  spectral  region, 
thermal  radiation  dominates  and  energy  emitted  by  objects  would  be  function  of  their 
temperature  and  emissivity.  Thus,  the  resulting  video  output  would  be  a  thermal  map  of  the 
forward  road  scene  where  hotter  objects  appear  brighter  than  cooler  objects. 

a.  LWIR  Camera. 

b.  Long  focal  length  lens  with  smaller  FOV. 

c.  Weight. 

d.  Environmental  consideration. 

e.  Real-time  monochrome  video  signal  output  from  camera. 

f.  Higher  thermal  sensitivity. 

g.  Faster  time  response  or  shorter  integration  time. 

h.  Real-time  video  display. 

i.  Mounting  bracket  required. 

j .  B  attery  powered . 

k.  Able  to  operate  in  intended  environment. 


6-10 


RCC  Special  Report:  Development  Of  Electro-Optical  (EO)  Standard  Processes  for  Application,  November  2011 


6.6.4  Airborne  Surveillance.  On  the  Level  I,  the  majority  of  the  airborne  surveillance  tasks 
would  be  of  EO/IR  imaging  in  down-looking  mode.  The  imaging  tasks  include  real-time  or  near 
real-time  battlefield  target  recognition  and  tracking,  facility  monitoring,  environmental 
observation,  thermal  plume  detection,  etc.  Depending  on  the  specific  tasks,  these  applications 
might  need  a  MWIR  or  LWIR  camera.  The  general  requirement  for  the  instrumentation  is 
compact  with  small  footprint  and  light  weight.  The  capability  of  measurement  under  all  weather 
conditions,  etc. 

6.7  Examples:  Level  II 

6.7.1  Target  Surface  Characterization.  Target  surface  characterization  includes  measurements 
on  surface  emissivity,  radiance,  reflectance,  transmittance,  polarization  effect,  surface  roughness, 
etc.  These  applications  in  general  require  cooled  IR  camera  with  appropriate  optics  such  as 
spectral  filters,  neutral  density  filters,  polarizers.  The  parameters  measured  are  functions  of 
angles,  temperature,  and  wavelengths. 

6.7.2  Chemical  and  Biological  Warfare  Agent  Remote  Sensing.  Most  molecules  and  organic 
compounds  possess  spectral  features  in  the  IR  that  can  be  used  for  their  detection  and 
identification.  Thus,  an  IR  camera  and  spectrometer  with  moderate  resolution  can  be  used  to 
detect  organic  compound  and  plums  that  may  be  related  to  toxic  gases,  pollutants,  chemical  and 
biological  warfare  agents,  drug  manufacturing  byproducts,  volcanic  eruption,  etc. 

6.8  Examples:  Level  III 

6.8.1  IR  Range  Measurement.  This  example  involves  target  characterization  inside  an 
environment  chamber  or  range.  The  parameters  measured  include  effective  (spectrally  weighted) 
target  emissivity,  reflectivity,  and  radiance.  The  experimental  parameters  are  target  temperature, 
target  elevation  and  azimuth  angles,  zone  temperatures  of  the  IR  range. 

Target  temperature  control  is  realized  by  either  running  chilled  coolant  through  its  body 
or  by  heating  up  its  temperature  using  embedded  electrical  heaters.  The  key  requirement  for 
target  temperature  control  is  to  have  uniform  temperature  distribution  over  the  whole  volume. 

Liquid  nitrogen  cooled  LWIR  camera  with  appropriate  spectral  filters  and  a  lens  that 
defined  a  smaller  FOV  are  used  for  the  measurement.  Two  different  blackbodies  (NIST 
traceable)  were  used  to  absolutely  calibrate  the  IR  camera.  Temperatures  of  the  blackbody  were 
chosen  to  encompass  as  much  as  possible  the  span  in  radiance  coming  from  the  potential  target 
or  sample  to  be  measured  under  different  environmental  and  viewing  conditions. 

Since  the  IR  camera  measures  effective  quantities,  coefficients  such  as  emissivity, 
reflectivity,  and  transmittance  are  converted  into  effective  quantities.  This  is  done  by  summing 
the  product  of  the  desired  spectral  quantity  and  the  spectral  responsivity  of  the  sensor  (including 
the  effect  of  spectral  responsivity  of  any  filter  or  filters  used),  and  then  dividing  the  result  by  the 
sum  of  the  spectral  responsivity. 

Measurements  for  target  surface  radiance  are  carried  out  in  an  environment  chamber. 

The  chamber  can  be  used  to  simulate  the  IR  energy  an  object  would  encounter  in  the  atmosphere 
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at  an  altitude  as  high  as  35000  feet.  This  is  accomplished  by  heating  or  cooling  five  different 
zones  inside  the  chamber.  In  the  measurements,  the  environment  chamber  temperature  zones  are 
set  to  simulate  a  particular  IR  environment  at  1000  or  5000  feet.  For  each  zone,  the  temperatures 
are  the  chamber  zone  temperature  which  yields  the  minimum  RMS  difference  between  the 
simulated  IR  chamber  radiance  (simulated  using  BOXMATCH  and  Moderate  Resolution 
Atmospheric  Transmission  (MODTRAN))  radiance  for  the  viewing  angles  corresponding  to  a 
particular  zone.  Modeling  is  needed  to  compare  with  the  measured  results. 

6.8.2  Bench  Mark  Thermal  Target.  In  this  example,  the  target  surfaces  are  either  flat  or 
cylindrical.  IR  surface  emissivity  is  measured  with  IR  imagers.  The  assumption  is  that  a  single 
Lambertian  source  is  giving  rise  to  radiant  power  and  that  the  power  is  being  detected  by  a 
sensor  some  distance  away  from  the  source.  The  target  temperature  control  is  realized  by  either 
flowing  coolant  through  its  body  or  by  heating  up  its  body  using  embedded  electrical  heaters. 
Thermal  modeling  is  needed  to  compare  with  the  measurements. 

6.9  Example:  NIIRS  Criterion 

Based  on  the  NIIRS  criteria,  an  example  of  progression  in  spatial  detail  for  target 
characterization  is  listed  in  the  following  Table: 


TABLE  6-2.  EXAMPLE  CRITERION  PER  NIIRS  LEVEL 

Level 

Examples  for  Target  Characterization 

0 

Interpretability  of  the  imagery  is  precluded  by  obscuration  or  very  poor  resolution* 

1 

Able  to  detect  a  medium-sized  facility  such  as  shipyard  or  a  factory  or  a  township 

2 

Able  to  detect  individual  target  such  as  big  buildings,  large  air  fields,  etc. 

3 

Able  to  detect  moving  targets  along  road  or  river 

4 

Identify  target  from  mixed  background  scenes  such  as  building,  field,  river,  forest,  etc. 

5 

Able  to  identify  the  type  of  the  target  such  as  a  car,  a  ship,  or  an  airplane,  etc. 

6 

Able  to  identify  the  geometry  or  the  shape  or  the  body  style  of  a  target 

7 

Able  to  identify  small  features  on  target  surface  such  as  bolts  or  holes,  etc. 

8 

Able  to  identify  target  surface  structure  details  such  as  patterns  or  textures 

9 

Able  to  extract  target  surface  roughness  information 

*  As  defined  in  MIL  Standard  150A(1),  the  concept  of  being  able  to  recognize  three  bars  as  a 
measure  of  image  quality  in  terms  of  resolution. 

Note  that  NIIRS  criteria  have  accounted  for  all  the  following  factors  that  impact  image 
interpretability: 

a.  Image  scale  (such  as  photographic  scale). 

b.  Distance  between  target  and  detector. 

c.  Signal  noise  ratio. 

d.  Imaging  system  characteristics  such  as  the  quality  of  the  optics  and  the  performance 
of  the  FPA  detector. 
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e.  Environmental  conditions  such  as  sun  angle,  atmospheric  transmission  and  haze. 

f.  Other  conditions  such  as  film  duplication  process,  video  monitor  quality,  etc. 

6.10  Common  Instrumentation  Nomenclature 

Figures  of  merits,  operating  characteristics,  and  other  constraints  must  be  considered 
when  matching  detectors  to  specific  applications.  The  following  are  some  of  common 
nomenclatures: 

a.  Responsivity  (power,  temporal,  and  spectral). 

b.  Noise  equivalent  power  (NEP). 

c.  Detectivity. 

d.  Specific  detectivity. 

e.  Noise  equivalent  irradiance. 

f.  Time  constant. 

g.  Quantum  efficiency. 
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CHAPTER  7 

DRAFT  HANDBOOK  FOR  ELECTRO-OPTICAL  (EO)  MEASUREMENTS 

This  chapter,  designed  as  a  draft  handbook,  is  fundamentally  based  on  a  document 
entitled.  Handbook  of  Electro-Optical  Standard  for  Applications  (EOSPA),  delivered  23  April 
2007  to  the  SMSG-18  task  lead,  Mr.  Jeffrey  W.  Burks  (EO/IR  Laboratory,  Code  SNS  (now  Code 
RYS)  at  the  Air  Force  Research  Laboratory  (  at  Wright-Patterson  AFB.  The  document  was 
prepared  under  contract  by  EO/IR  Remote  Sensing  Group,  Mission  Research  and  Technical 
Service  Division,  Alliant  Techsystems  Inc.  That  deliverable  document,  and  this  report,  is 
principally  a  narrative  based  on  ANSI  Z-540.1-1994  and  adapted  to  the  field  of  EO/IR 
measurements.  The  adaptation  includes  examples  and  narrative  guidance  for  most  of  the 
evaluation  criteria. 

1  PARTI:  INTRODUCTION 

As  part  of  the  RCC  effort  to  serve  the  technical  and  operational  needs  of  member  test 
ranges  or  facilities,  the  SMSG  was  formed  to  document  signature  measurement  standards  across 
the  electromagnetic  spectrum,  and  specifically  in  the  following  areas:  radar,  millimeter  wave, 
infrared,  laser,  visible,  ultraviolet,  seismic,  acoustic,  and  magnetic.  This  handbook  is  based  in 
part  on  RCC  804-01  and  IRIG  Standard  RCC  802-98,  Tactical  Missile  Signatures  Measurement 
Standard  and  Definitions,  and  is  intended  to  assist  SMSG  members  in  EO  and  infrared  IR 
signature  measurements  by  documenting  standard  procedures.  The  overall  structure  and 
languages  used  in  the  Handbook  are  in  parallel  with  that  in  the  Handbook  for  the  Interpretation 
and  Application  of  ANSI/NCSL  Z540-1-1994  (Reference  1.2a). 

1.1  Scope 

This  handbook  documents  standards  for  characterizing  the  measurement  capabilities  of 
EO/IR  measurement  systems  and  helps  establish  a  demonstration  program  that  verifies  and 
quantifies  the  improvement  in  measurement  capability  that  will  result  from  implementing 
documented  standards. 

The  standards  defined  include: 

a.  Instrument  characterization  and  calibration  procedures. 

b.  Standard  data  reduction  procedures  to  transform  raw  data  to  engineering  units. 

c.  Measurement  uncertainty  analysis. 

d.  Standard  terminology  and  units  of  measure. 

This  draft  provides  standard  for  planning,  executing,  and  reporting  signature 
measurements.  It  also  describes  the  resources  available  to  the  IR  range  community  and 
recommends  procedures  to  help  testers  make  efficient,  effective  use  of  instrumentation. 

Compliance  with  the  standard  and  use  of  the  recommended  procedures  will  result  in  well- 
documented  calibration  and  test  data  sets  available  to  all  users  within  the  SMSG  Group. 
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1.2  References 

a.  Handbook  for  the  Interpretation  and  Application  of  ANSI/NCSL  Z540- 1- 1994. 

b.  The  International  vocabulary  of  basic  and  general  terms  in  metrology  (VIM). 

c.  ISO/IEC  Guide  2:  1986,  General  terms  and  their  definitions  concerning  standardization 
and  related  activities. 

d.  ISO/IEC  Guide  25:  1990,  General  requirements  for  the  competence  of  calibration  and 
testing  laboratories. 

e.  ISO  Guide  30:  1981,  Terms  and  definitions  used  in  connection  with  reference  materials. 

f.  ISO  Guide  10012-1. 

g.  MIL-STD-45662A:  1988,  Calibration  Systems  Requirement. 

h.  ISO  8402:  1986,  Quality  -  Vocabulary. 

i.  ISO  Guide  to  the  expression  of  uncertainty  in  measurement. 

1.3  Definitions  Relevant  definitions  used  in  the  Handbook  are  quoted  in  the  following 
sections  with  notes. 

1.3.1  Calibration.  The  set  of  operations  which  establish,  under  specified  conditions,  the 
relationship  between  values  indicated  by  a  measuring  instrument  or  measuring  system,  and  the 
corresponding  standard  or  known  values  derived  from  the  standard. 

Note:  (Reference  1.2b  -  paragraph  6.11,  with  note  4  added): 

a.  The  result  of  a  calibration  permits  the  estimation  of  errors  of  indication  of  the 
measuring  instrument,  measuring  system,  or  the  assignment  of  values  to  marks  on 
arbitrary  scales. 

b.  A  calibration  may  also  determine  other  metrological  properties. 

c.  The  result  of  a  calibration  may  be  recorded  in  a  document,  sometimes  called  a 
calibration  certificate  or  a  calibration  report. 

d.  The  result  of  a  calibration  is  sometimes  expressed  as  a  calibration  factor,  or  as  a  series 
of  calibration  factors  in  the  form  of  a  calibration  curve. 
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1 .3.2  Calibration  Certificate  or  Report.  Document  prepared  that  presents  calibration  results 
and  other  information  relevant  to  a  calibration  (Reference  1.2c  -  12.3,  amended). 

1.3.3  Calibration  Method.  Defined  technical  procedure  for  performing  a  calibration  or 
verification  (Reference  1.2d  -  3.9,  amended) 

1.3.4  Certified  Reference  Material  (CRM).  A  reference  material  one  or  more  of  whose 
property  values  are  certified  by  a  technically  valid  procedure,  accompanied  by  or  traceable  to  a 
certificate  or  other  documentation  which  is  issued  by  a  certifying  body  (Reference  1.2e  -  2.2). 

1.3.5  Inter  laboratory  Comparisons.  Organization,  performance  and  evaluation  of  calibrations 
on  the  same  or  similar  items  by  two  or  more  laboratories  in  accordance  with  predetermined 
conditions  (Reference  1.2c  -  12.5) 

1.3.6  International  (Measurement)  Standard.  A  standard  recognized  by  an  international 
agreement  to  serve  internationally  as  the  basis  for  fixing  the  value  of  all  other  standards  of  the 
quantity  concerned  (Reference  1.2b  -  paragraph  6.2). 

1.3.7  Influence  Quantity.  A  quantity  which  is  not  the  subject  of  the  measurement  but  which 
influences  the  value  of  the  measurand  or  the  indication  of  the  measuring  instrument 
(Reference  1.2b  -  paragraph  2.7,  amended) 

Examples:  ambient  temperature;  pressure;  humidity  level;  frequency  of  an  alternating  measured 
voltage. 

1.3.8  Laboratory/Calibration  Laboratory .  Body  that  calibrates  or  performs  calibrations  and 
verifications 

Notes  (Reference  1 .2d  -  paragraph  3.1,  amended): 

In  cases  where  a  laboratory  forms  part  of  an  organization  that  carries  out  other  activities  besides 
calibration,  the  term  "laboratory"  refers  only  to  those  parts  of  that  organization  that  are  involved 
in  the  calibration  process. 

As  used  herein,  the  term  "laboratory"  refers  to  a  body  that  carries  out  calibration  at  or  from  a 
permanent  location,  at  or  from  a  temporary  facility,  or  in  or  from  a  mobile  facility. 

1.3.9  Limits  of  Permissible  Error  (of  a  Measuring  Instrument).  The  extreme  values  of  an  error 
permitted  by  specifications,  regulations,  etc.  for  a  given  measuring  instrument  (Reference  1.2b  - 
paragraph  5.2,  with  added  note) 

Note:  This  term  is  frequently  referred  to  as  "tolerance"  in  the  United  States. 

1.3.10  Measurand.  A  quantity  subjected  to  measurement  (Reference  1.2b  -  paragraph  2.6, 
amended) 

Note:  As  appropriate,  this  may  be  the  "measured  quantity"  or  the  "quantity  to  be  measured." 
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1.3.11  Measurement.  The  set  of  operations  having  the  object  of  determining  the  value  of  a 
measurand  (Reference  1.2b  -  paragraph  2.1,  amended,  without  note). 

1.3.12  Measurement  Assurance.  Measurement  assurance  is  a  technique  that  may  include,  but  is 
not  limited  to: 

a.  Use  of  good  experimental  design  principles  so  the  entire  measurement  process,  its 
components,  and  relevant  influence  factors  can  be  well  characterized,  monitored  and 
controlled; 

b.  Complete  experimental  characterization  of  the  measurement  process  uncertainty 
including  statistical  variations,  contributions  from  all  known  or  suspected  influence 
factors,  imported  uncertainties,  and  the  propagation  of  uncertainties  throughout  the 
measurement  process;  and 

c.  Continuously  monitoring  the  performance  and  state  of  statistical  control  of  the 
measurement  process  with  proven  statistical  process  control  techniques  including  the 
measurement  of  well  characterized  check  standards  along  with  the  normal  workload 
and  the  use  of  appropriate  control  charts. 

1.3.13  Measurement  Standard.  A  material  measure,  measuring  instrument,  reference  material  or 
system  intended  to  define,  realize,  conserve  or  reproduce  a  unit  or  one  or  more  known  values  of 
a  quantity  to  serve  as  a  reference  (Reference  1.2b  6.1,  without  note). 

1.3.14  Measuring  and  Test  Equipment.  All  of  the  measuring  instruments,  measurement 
standards,  reference  materials,  and  auxiliary  apparatus  that  are  necessary  to  perform  a 
measurement.  This  term  includes  measuring  equipment  used  in  the  course  of  testing  and 
inspection,  as  well  as  that  used  in  calibration  (Reference  1.2f  —  paragraph  3.2,  amended). 

Note:  In  the  context  of  this  Standard,  the  term  "measuring  and  test  equipment"  is  taken  to 
encompass  "measuring  instruments"  and  "measurement  standards".  Moreover,  a  "reference 
material"  is  considered  a  type  of  "measurement  standard". 

1.3.15  Measuring  Instrument.  A  device  intended  to  make  a  measurement,  alone  or  in 
conjunction  with  supplementary  equipment  (Reference  1.2b  4.1) 

1.3.16  Mutual  Consent  Standard.  An  artifact  or  process  that  is  used  as  a  de  facto  standard  by 
mutual  consent  of  the  supplier  and  customer  when  no  recognized  U.S.  national  standard  is 
available  (Reference  1.2g  -  3.2,  amended) 

1.3.17  National  (Measurement)  Standard.  A  standard  recognized  by  an  official  national  decision 
to  serve,  in  a  country,  as  the  basis  for  fixing  the  value  of  all  other  standards  of  the  quantity 
concerned  (Reference  1.2b  -  paragraph  6.3) 

1.3.18  Proficiency  Testing.  Determination  of  the  laboratory  calibration  performance  by  inter 
laboratory  comparisons  or  other  means  (Reference  1.2c  -  paragraph  12.6,  amended). 

1.3.19  (Quality)  Audit.  A  systematic  and  independent  examination  to  determine  whether  quality 
activities  and  related  results  comply  with  planned  arrangements  and  whether  these  arrangements 
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are  implemented  effectively  and  are  suitable  to  achieve  objectives  (Reference  1.2h  - 
paragraph  3.10,  with  note  1  only). 

Note:  The  quality  audit  typically  applies  to,  but  is  not  limited  to,  a  quality  system  or  elements 
thereof,  to  processes,  to  products,  or  to  services.  Such  an  audit  is  often  called  a  quality  system 
audit,  a  process  quality  audit,  a  product  quality  audit,  or  a  service  quality  audit. 

1.3.20  Quality  Manual.  A  document  stating  the  quality  policy,  quality  system  and  quality 
practices  of  an  organization  (Reference  1.2d  -  paragraph  3.10). 

Note:  The  quality  manual  may  call  up  other  documentation  relating  to  the  laboratory's  quality 
arrangements. 

1.3.21  Quality  System.  The  organizational  structure,  responsibilities,  procedures,  processes  and 
resources  for  implementing  quality  management  (Reference  1.2h  -3.8,  without  notes) 

1.3.22  (Quality  System)  Review.  A  formal  evaluation  by  management  of  the  status  and 
adequacy  of  the  quality  system  in  relation  to  quality  policy  and  new  objectives  resulting  from 
changing  circumstances  (Reference  1.2h  -  paragraph  3.12) 

1.3.23  Reference  Material.  A  material  or  substance  of  which  one  or  more  properties  are 
sufficiently  well  established  to  be  used  for  the  calibration  of  an  apparatus,  the  assessment  of  a 
measurement  method,  or  for  assigning  values  to  materials  (Reference  1.2e  -  paragraph  2.17, 
amended,  without  notes) 

1.3.24  Reference  Standard.  A  standard,  generally  of  the  highest  metrological  quality  available 
at  a  given  location,  from  which  measurements  made  at  that  location  are  derived  (Reference  1.2b 
-  paragraph  6.6). 

1.3.25  Requirement.  A  translation  of  the  needs  into  a  set  of  individual  quantified  or  descriptive 
specifications  for  the  characteristics  of  an  entity  in  order  to  enable  its  realization  and  examination 
(Reference  1.2d  -  paragraph  3.16) 

1.3.26  Traceability.  The  property  of  a  result  of  a  measurement  whereby  it  can  be  related  to 
appropriate  standards,  generally  national  or  international  standards,  through  an  unbroken  chain 
of  comparisons  (Reference  1.2b  -  paragraph  6.10,  amended). 

1.3.27  Uncertainty  of  Measurement.  Parameter,  associated  with  the  result  of  a  measurement  that 
characterizes  the  dispersion  of  the  values  that  could  reasonably  be  attributed  to  the  measurand 
(Reference  1.2b  -  paragraph  3.9). 

Notes: 

a.  The  parameter  may  be,  for  example,  a  standard  deviation  (or  a  given  multiple  of  it), 
or  the  half-width  of  an  interval  having  a  stated  level  of  confidence. 

b.  Uncertainty  of  measurement  comprises,  in  general,  many  components.  Some  of  these 
components  may  be  evaluated  from  the  statistical  distribution  of  the  results  of  series 
of  measurements  and  can  be  characterized  by  experimental  standard  deviations.  The 
other  components,  which  can  also  be  characterized  by  standard  deviations,  are 
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evaluated  from  assumed  probability  distributions  based  on  experience  or  other 
information. 

c.  It  is  understood  that  the  result  of  the  measurement  is  the  best  estimate  of  the  value  of 
the  measurand,  and  that  all  components  of  uncertainty,  including  those  arising  from 
systematic  effects,  such  as  components  associated  with  corrections  and  reference 
standards,  contribute  to  the  dispersion. 

d.  This  definition  is  that  of  the  “Guide  to  the  Expression  of  Uncertainty  in 
Measurement”  in  which  its  rationale  is  detailed. 

1.3.28  Verification  Evidence  by  calibration  that  specified  requirements  have  been  met 
(Reference  1.2d  -  paragraph  3.8,  amended) 

Notes: 

a.  In  connection  with  the  management  of  measuring  equipment,  verification  provides  a 
means  for  checking  that  the  deviations  between  values  indicated  by  a  measuring 
instrument  and  corresponding  known  values  are  consistently  smaller  than  the  limits  of 
permissible  error  defined  in  a  standard,  regulation  or  specification  peculiar  to  the 
management  of  the  measuring  equipment. 

b.  The  result  of  verification  leads  to  a  decision  either  to  restore  to  service,  or  to  perform 
adjustments,  or  to  repair,  or  to  downgrade,  or  to  declare  obsolete.  In  all  cases, 
documentation  of  the  verification  performed  is  kept  on  the  measuring  instrument's 
individual  record. 

c.  The  term  "verification"  as  defined  in  this  Standard  is  frequently  referred  to  as 
"calibration"  in  the  United  States. 

1.3.29  Approved  Signatory  A  person  who  is  recognized  by  an  accredited  body  as  competent  to 
sign  accredited  laboratory  calibration  reports  (Reference  1.2c  -  paragraph  16.9,  amended) 

2  PART  II:  GENERAL  REQUIREMENTS  FOR  THE  COMPETENCE  OF 
CALIBRATION  LABORATORIES 

The  following  discussions  apply  to  calibration  laboratories  in  the  development  and 
implementation  of  their  quality  system. 

2.1  Laboratory  Organization  and  Management 

2.1.1  Legal  Identity.  The  laboratory  shall  be  legally  identifiable.  It  shall  be  organized  and 
shall  operate  in  such  a  way  that  its  permanent,  temporary,  and  mobile  facilities  meet  the 
requirements  of  EOSPA. 

2. 1.1.1  Interpretive  Guidance.  The  laboratory  is  normally  an  established  business  or 
corporation,  or  an  identifiable  division  of  a  business  or  corporation,  which  meets  the  applicable 
legal  requirements  of  the  governmental  jurisdiction  in  which  it  conducts  business.  The 
management  and  organization  of  the  laboratory  should  be  structured  to  ensure  that  applicable 
facilities  conform  to  this  Standard.  Legal  identifiability  may  aid  in  addressing  issues  of 
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liability/accountability,  uniqueness,  composition/scope,  and  independence  of  operation.  Legal 
identification  may  also  aid  in  the  prevention  of  misapplication  of  the  Standard  to  parts  of  the 
organization  which  do  not  truly  comply. 

2. 1.1.2  Example(s).  A  laboratory  may  be  a  department,  division  or  subdivision  of  a  business 
or  corporation  that  supplies  products  or  services  in  addition  to,  or  other  than,  calibration  services, 
or  it  may  be  a  business  or  corporation  whose  sole  activity  is  to  provide  calibration  services.  A 
laboratory  may  operate  as  a  function  within  a  department  of  the  Federal  Government,  such  as  the 
Department  of  Energy  or  the  Department  of  Defense,  or  it  may  be  a  Weights  and  Measures 
Laboratory  of  a  state  which  is  frequently  found  within  the  jurisdiction  of  the  state's  Department 
of  Agriculture. 

2.1.2  Managerial  Staff.  The  laboratory  shall  have  managerial  staff  with  the  authority  and 
resources  needed  to  discharge  their  duties. 

2. 1.2.1  Interpretive  Guidance.  Laboratory  management  needs  the  authority  to  plan,  organize, 
direct  and  control  to  assure  quality  and  protect  integrity  of  results.  Laboratory  management 
needs  the  support  of  senior  management  in  terms  of  budget,  equipment,  facilities,  and  people. 

2. 1.2.2  Excimple(s).  Many  organizations  give  the  laboratory  manager  the  authority  to  select 
laboratory  personnel,  measurement  equipment,  laboratory  location,  etc.  as  well  as  inputs  to  the 
budget  process.  Indicators  such  as  increased  backlog,  missed  delivery  dates,  excessive  errors, 
etc.,  are  often  signs  of  inadequate  resources  and/or  authority. 

2.1.3  Personnel.  The  laboratory  shall  have  arrangements  to  ensure  that  its  personnel  are 
free  from  any  undue  pressures  that  might  adversely  affect  the  quality  of  their  work. 

2. 1.3.1  Interpretive  Guidance.  Laboratory  personnel  should  be  insulated  from  any  work 
related  pressures  that  would  compromise  the  quality  of  work.  The  source  of  undue  pressure  may 
be  internal  due  to  management  pressure,  deadlines,  etc.  or  external  due  to  customer  complaints, 
priority  requests  etc. 

2. 1.3.2  Example(s).  Communications  (priority  requests,  complaints,  status  inquiries,  etc.) 
could  be  directed  through  supervision  or  administrative  personnel.  The  existence  of  company 
ethics  programs,  skip-level  management  interviews,  company  ombudsman  programs,  etc.  may 
also  provide  avenues  of  coordination  that  preclude  adverse  effects  resulting  from  commercial 
pressures.  An  independent  lab  might  use  employee  performance  goals  that  are  based  more  on 
the  quality  of  work  rather  than  the  quantity. 

2.1.4  Organization  Structure.  The  laboratory  shall  be  organized  in  such  a  way  that 
confidence  in  its  independence  of  judgment  and  integrity  is  maintained  at  all  times. 

2. 1.4.1  Interpretive  Guidance.  Whether  a  laboratory  operates  as  part  of  a  larger 
organization,  government  agency,  or  as  a  stand-alone  business,  it  should  be  operated  so  that 
quantity  does  not  influence  quality. 

2. 1.4. 2  Example(s).  A  laboratory  may  organizationally  report  to  the  department  responsible 
for  quality  of  the  delivered  product  rather  than  the  production  department. 
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2.1.5  Documentation.  The  laboratory  shall  specify  and  document  the  responsibility,  authority, 
and  interrelations  of  all  personnel  who  manage,  perform,  or  verify  work  influencing  the  quality 
of  calibrations. 

2. 1.5.1  Interpretive  Guidance.  The  laboratory  should  have  an  organization  chart  and  job 
descriptions  for  the  entire  calibration  laboratory.  Responsibilities  and  authorities  should  be 
clearly  defined. 

2. 1.5.2  Example(s).  Job  descriptions  may  be  either  general  or  detailed  depending  on  the  size, 
complexity,  flexibility,  and  functions  of  the  organization.  The  important  issue  is  that  all 
concerned  parties  must  be  able  to  understand  the  roles,  responsibilities,  and  authorities  associated 
with  their  assignments  and  that  this  information  is  documented. 

2.1.6  Supervision.  The  laboratory  shall  provide  supervision  by  persons  familiar  with  the 
calibration  methods  and  procedures,  the  objective  of  the  calibration  and  the  assessment  of  the 
results.  Management  practices  shall  be  such  as  to  ensure  adequate  supervision. 

2. 1.6.1  Interpretive  Guidance.  Individuals  assigned  direct  supervisory  responsibilities  over 
laboratory  personnel  should  be  knowledgeable  in  the  methods,  practices  and  procedures  of  the 
calibrations  being  performed  and  of  assessing  calibration  results.  The  responsibilities  of  these 
supervisory  personnel  should  not  be  so  extensive  as  to  limit  their  effectiveness. 

2. 1.6.2  Example(s).  The  laboratory  supervisor  should  have  the  necessary  training/skills  in 
related  disciplines  sufficient  to  understand  the  measurements  being  made,  assess  the  results,  and 
make  decisions  that  affect  the  quality  of  the  results. 

2.1.7  Technical  Manager.  The  laboratory  shall  have  a  technical  manager  (however  named) 
who  has  overall  responsibility  for  the  technical  operations. 

2. 1.7.1  Interpretive  Guidance.  There  should  be  an  individual  designated  whose  job 
description  includes  the  authority  to  make  decisions  regarding  the  technical  (standard  selection, 
calibration  method  selection,  technical  qualifications  of  personnel,  etc.)  aspects  of  laboratory 
operations. 

2. 1.7. 2  Example(s).  As  one  example,  a  laboratory  might  assign  a  senior  engineer,  senior 
technician,  chief  metrologist  or  similar  title  to  the  position  of  technical  manager,  having  the 
responsibility  for  the  technical  validity  of  all  aspects  of  laboratory  operations.  As  another 
example,  a  laboratory  might  have  a  single  person  fill  several  roles,  one  of  which  is  to  serve  as 
technical  manager. 

2.1.8  Quality  Manager.  The  laboratory  shall  have  a  quality  manager  (however  named)  who 
has  responsibility  for  the  quality  system  and  its  implementation.  The  quality  manager  shall  have 
direct  access  to  the  highest  level  of  management  at  which  decisions  are  taken  on  laboratory 
policy  or  resources,  and  to  the  technical  manager.  In  some  laboratories,  the  quality  manager  may 
also  be  the  technical  manager  or  deputy  technical  manager. 
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2. 1.8.1  Interpretive  Guidance.  Responsibility  for  the  quality  system  of  the  laboratory  should 
be  assigned  to  an  individual  who  is  free  of  pressures,  which  may  conflict,  with  the  quality  of 
calibrations. 

2. 1.8.2  Example(s).  In  a  larger  organization,  the  quality  manager  might  report  to  the 
manager/director  of  the  Quality  Assurance  department  or  its  equivalent.  In  a  smaller 
independent  laboratory,  the  quality  manager  might  report  to,  or  even  be,  the  laboratory  technical 
manager,  president,  or  owner. 

2.1.9  Designated  Alternates.  The  laboratory  shall  designate  alternates  in  case  of  absence  of 
the  technical  or  quality  manager. 

2. 1.9.1  Interpretive  Guidance.  Individuals  who  have  the  required  knowledge  and  expertise  to 
assume  the  duties  of  the  technical  or  quality  manager  should  be  assigned  as  alternates  to  provide 
continuity  of  operation  during  absences  of  the  technical  or  quality  manager. 

2. 1.9.2  Example(s).  An  organizational  chart  or  other  suitable  documentation  that  identifies 
alternates  for  the  technical  and  quality  managers  might  be  posted  or  otherwise  made  available. 

2.1.10  Policy  and  Procedures.  The  laboratory  shall  have,  where  relevant,  have  documented 
policy  and  procedures  to  ensure  the  protection  of  customer's  confidential  information  and 
proprietary  rights. 

2.1.10.1  Interpretive  Guidance.  A  laboratory  should  identify  any  customer  information  or 
material  that  is  considered  confidential  or  proprietary  and  have  appropriate  policies  and 
procedures  in  place  to  protect  the  customer's  interest. 

2.1.10.2  Example(s).  Identification  of  the  need  to  protect  information  should  logically  be  a 
requirement  of  the  contract  (real  or  implied)  between  the  laboratory  and  its  customer(s). 

2.1.11  Measurement  Assurance  Programs.  The  laboratory  shall,  where  appropriate, 
participate  in  inter  laboratory  comparisons  and  proficiency  testing  programs. 

2.1.11.1  Interpretive  Guidance.  Laboratories  are  encouraged  to  participate  in  measurement 
assurance  programs  (MAPs)  or  other  types  of  inter  comparison  programs  between  laboratories 
which  provide  increased  confidence  in  processes  and  procedures.  The  appropriateness  of  such 
programs  may  be  determined  by  the  laboratory  and/or  customer  needs. 

2.1.11.2  Example(s).  Inter  comparison  type  programs  are  useful  and  effective  tools  for 
evaluating  uncertainties  in  many  measurement  processes.  Such  programs  are  especially 
appropriate  when  intrinsic  standards  are  employed  or  where  no  traceability  to  a  national  standard 
is  available.  Where  intrinsic  standards  are  used,  the  laboratory  should  demonstrate  by 
measurement  assurance  techniques,  inter  laboratory  comparisons,  or  other  suitable  means,  that  its 
intrinsic  standard  measurement  results  are  correlated  with  those  of  national  or  international 
standards.  Where  traceability  to  national  standards  is  not  applicable,  the  laboratory  should 
provide  evidence  of  correlation  of  results  by  participation  in  a  suitable  program  of  inter 
laboratory  comparisons. 


7-9 


RCC  Special  Report:  Development  Of  Electro-Optical  (EO)  Standard  Processes  for  Application,  November  2011 


2.2  Quality  System,  Audit,  and  Review. 

2.2.1  Quality  System.  The  laboratory  shall  establish  and  maintain  a  quality  system  appropriate 
to  the  type,  range  and  volume  of  calibration  activities  it  undertakes.  The  elements  of  this  system 
shall  be  documented.  The  quality  documentation  shall  be  available  for  use  by  the  laboratory 
personnel.  The  laboratory  shall  define  and  document  its  policies  and  objectives  for,  and  its 
commitment  to,  good  laboratory  practice  and  quality  of  calibration  services.  The  laboratory 
management  shall  ensure  that  these  policies  and  objectives  are  documented  in  a  quality  manual 
and  communicated  to,  understood,  and  implemented  by  all  laboratory  personnel  concerned.  The 
quality  manual  shall  be  maintained  current  under  the  responsibility  of  the  quality  manager. 

2. 2. 1.1  Interpretive  Guidance.  The  laboratory  should  have  a  quality  system  documented  in  a 
quality  manual  that  defines  the  policies  and  practices  which  must  be  followed  in  the  conduct  of 
its  business.  Laboratory  management  is  responsible  for  ensuring  that  all  laboratory  personnel  are 
aware  of,  understand,  and  implement  the  elements  of  the  quality  system.  The  quality  manager  is 
responsible  for  maintaining  the  quality  manual. 

2.2.2  Quality  Manual.  The  quality  manual  and  related  documentation  shall  state  the 
laboratory's  policies  and  operational  procedures  established  in  order  to  meet  the  requirements  of 
this  Standard. 

2.2.2. 1  Interpretive  Guidance.  The  phrase  "quality  manual  and  related  documentation"  is  a 
direct  reference  to  the  quality  documentation  defined  in  paragraph  2.2.1  as  being  required  to  "be 
available  for  use  by  the  laboratory  personnel".  The  quality  manual  should  contain  "policies  and 
objectives  for,  and  commitment  to,  good  laboratory  practice  and  quality  calibration  services.” 
These  policies  and  objectives  should  reference  the  quality  documentation  containing  the 
procedures  and  practices  to  be  followed  to  ensure  the  policies  and  objectives  are  met.  The 
quality  manual  and  related  documentation  shall  also  contain  the  following  list  of  subjects  which 
should  be  addressed  in  the  quality  manual  and  related  documentation: 

1)  A  quality  policy  statement,  including  objectives,  by  top  management,  expressing 
the  commitment  to  and  intent  of  management  and  staff  to  provide  sound  and 
reliable  services  to  its  customers; 

2)  The  organization  and  management  structure  of  the  laboratory,  its  place  in  any 
parent  organization,  related  organization  charts,  outline,  or  description  of  the 
organizational  structure  of  the  laboratory  and  its  location  within  a  parent 
organization,  if  applicable,  should  be  included.  A  chain  of  command  should  be 
established  to  identify  responsibilities  and  personnel  who  have  critical  positions 
within  the  laboratory. 

3)  The  relations  between  management,  technical  operations,  support  services  and  the 
quality  system  (A  description  of  how  the  various  organizational  components  of 
the  laboratory  interrelate  to  create  a  single  cohesive  laboratory  program  should  be 
included). 

4)  Procedures  for  control  and  maintenance  of  documentation:  The  procedures, 
responsibilities,  and  authorities  for  drafting,  changing,  approving,  and  issuing 
documents  needed  for  conducting  the  laboratory's  business  (procedures,  drawings, 
schedules,  work  plans,  job  orders,  test  methods,  etc.),  including  the  quality 
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manual  and  related  quality  documentation,  should  be  documented  and  referenced. 
The  control  of  documents  should  ensure  that: 

a.  Pertinent  issues  of  appropriate  documents  are  available  at  all  locations  where 
operations  essential  for  the  functioning  of  the  quality  system  are  performed. 

b.  Invalid  and/or  obsolete  documents  are  promptly  removed  from  all  points  of 
issue  or  use,  or  otherwise  assured  against  unintended  use. 

c.  Any  obsolete  documents  retained  for  legal  and/or  knowledge-reservation 
purposes  are  suitably  identified. 

5)  Job  descriptions  of  key  staff  and  reference  to  descriptions  of  other  staff:  Job 
descriptions  of  personnel  in  positions  of  responsibility;  i.e.,  those  who  are 
empowered  to  make  decisions  should  be  included.  Job  descriptions  of  all  other 
laboratory  personnel,  whose  duties  might  affect  the  quality  of  the  service 
provided,  should  be  referenced  as  to  where  details  might  be  found. 

6)  Identification  of  the  approved  signatories  of  the  laboratory  (where  this  concept  is 
appropriate):  An  approved  signatory  is  defined  as  an  individual  (not  a  position  or 
job  title)  who  is  recognized  by  an  accreditation  body  as  competent  to  sign 
accredited  laboratory  test  or  calibration  reports.  Approved  signatories  should  be 
persons  with  responsibility,  authority  and  technical  capability  within  the 
organization  for  the  results  produced.  In  selecting  approved  signatories,  the 
following  should  be  considered: 

a.  Qualification  and/or  experience; 

b.  Position  in  the  staff  structure  (should  be  technical  personnel  closely  involved 
in  the  day-to-day  operations  of  the  laboratory). 

c.  Familiarity  with  the  procedures  and  awareness  of  any  limitations  of  these 
procedures. 

d.  Ability  to  make  critical  evaluations  of  calibration  results  and  a  position  in  the 
staff  structure  which  makes  them  responsible  for  the  adequacy  of  results. 

e.  Knowledge  of  quality  assurance  procedures  employed  in  the  laboratory  and 
ability  to  take  appropriate  and  effective  corrective  actions,  when  required.  An 
approved  signatory  may  be  a  person  engaged  by  the  laboratory  as  a  consultant 
or  in  a  similar  capacity  as  long  as  all  requirements  for  competency  are  met. 

Note:  Paragraph  6  above  only  applies  to  accredited  laboratories.  Non- accredited 
laboratories  wishing  to  be  compliant  with  this  Standard  should  internally  designate 
approved  signatories,  following  the  same  guidelines  for  selection. 

7)  The  laboratory's  procedures  for  achieving  traceability  of  measurements: 

Reference  to  the  methods  and  procedures  used  to  establish  the  traceability  path,  to 
a  national  laboratory,  an  intrinsic  standard  or  other  acceptable  source,  should  be 
included. 

8)  The  laboratory's  scope  of  calibrations  and/or  verifications:  The  laboratory's  scope 
of  calibrations  describes  the  measurement  and/or  calibration  capabilities  of  the 
service  it  provides. 

9)  Arrangements  for  ensuring  that  the  laboratory  reviews  all  new  work  to  ensure  that 
it  has  the  appropriate  facilities  and  resources  before  commencing  such  work:  The 
quality  documentation  should  contain  procedures  which  ensure  the  laboratory  has 
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the  personnel,  facilities,  equipment,  standards,  and  technical  expertise  necessary 
to  handle  proposed  work  effectively  before  it  is  accepted.  Particular  attention 
should  be  paid  to  work  of  a  type  not  currently  being  performed  in  the  laboratory. 
Consideration  should  be  given  to: 

a.  The  scope  of  calibrations  normally  performed. 

b.  Available  resources  including  equipment,  staff,  and  space. 

c.  Workload. 

10)  Reference  to  the  calibration  and  verification  procedures  used: 
Calibration/verification  procedures  are  documented  in  accordance  with  the 
standard  and  are  considered  part  of  the  quality  documentation  of  a  laboratory. 
Reference  to  these  procedures  should  be  made. 

11)  Procedures  for  handling  calibration  and  verification  items:  Calibration  and 
verification  items  refers  primarily  to  customer  equipment  or  "device  under  test", 
that  is,  equipment  undergoing  calibration/verification.  Requirements  for 
documentation  of  these  procedures  are  defined  in  the  Standard  and  should  be 
referenced. 

12)  Reference  to  the  major  equipment  and  reference  measurement  standards  used:  A 
general  description  of  the  measurement  systems  and  measurement  standards  used 
by  the  laboratory  should  be  included.  This  provides  information  related  to  the 
laboratory's  scope  of  calibrations  and  method(s)  of  achieving  traceability. 

Detailed  requirements  for  the  care  and  usage  of  laboratory  equipment  are 
addressed  in  the  Standard. 

13)  Reference  to  procedures  for  calibration,  verification  and  maintenance  of 
equipment  used:  "Equipment  used"  refers  to  all  laboratory  equipment  (owned, 
rented  or  leased)  having  an  effect  on  the  results  of  the  calibration  services 
provided.  Detailed  requirements  for  the  care  of  laboratory  equipment  are  in  the 
Standard. 

14)  Reference  to  quality  assurance  practices  including  inter  laboratory  comparisons, 
proficiency-testing  programs,  use  of  reference  materials,  and  internal  quality 
control  schemes.  Quality  assurance  practices  are  used  to: 

a.  determine  agreements  between  laboratories. 

b.  demonstrate  compliance  to  certain  specifications. 

c.  indicate  that  the  measurement  processes  are  under  statistical  control. 

d.  demonstrate  traceability. 

e.  ensure  the  quality  of  results. 

Details  of  using  QA  practices  and  requirements  are  in  the  Standard. 

15)  Procedures  for  feedback  and  corrective  action  whenever  measurement 
discrepancies  are  detected,  or  departures  from  documented  policies  and 
procedures  occur:  Policies  and  procedures  should  describe  the  actions  to  be  taken 
when  it  is  discovered  that  an  error  has  occurred  or  procedures  were  not  followed. 
An  assessment  of  the  impact  (of  the  discrepancy  or  departure)  on  the  results  of 
measurements  and  a  determination  of  appropriate  corrective  action  should  be 
conducted.  This  requirement  pertains  to  discrepancies  or  unplanned  departures 
from  normal  operation. 

16)  The  laboratory  management  arrangements  for  exceptionally  permitting  departures 
from  documented  policies  and  procedures  or  from  standard  specifications: 
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Laboratory  management  is  responsible  for  ensuring  that  laboratory  policies  and 
procedures  are  followed.  The  quality  documentation  should  include  the  required 
process  to  be  followed  when  departing  from  these  policies  and  procedures. 

17)  Procedures  for  dealing  with  complaints:  A  section  of  the  Standard  contains  the 
requirements  for  dealing  with  complaints.  These  policies  and  procedures  should 
be  included  in  the  quality  documentation. 

18)  Procedures  for  protecting  confidentiality  and  proprietary  rights:  The  laboratory’s 
procedures  for  identifying  and  protecting  any  customer  information  or  material 
considered  confidential  or  proprietary  should  be  included  in  the  quality 
documentation. 

19)  Procedures  for  audit  and  review:  An  audit  is  a  detailed  evaluation  of  the 
compliance  with  the  quality  system.  It  attempts  to  find  out  if  the  system  is  in  fact 
working  the  way  it  is  documented.  A  review  is  a  comparison  of  the  system  with 
the  general  standard,  other  quality  needs,  and  the  general  business  of  the 
laboratory  to  determine  if  something  needs  to  be  changed  in  the  system  which 
will  end  up  with  new  or  revised  documentation.  These  are  separate  functions  and 
must  be  treated  separately  by  the  laboratory.  Requirements  for  audit  and  review 
procedures  are  defined  in  the  Standard  and  should  be  included  in  the  quality 
documentation. 

20)  A  statement  of  the  lab's  policy  for  establishing  and  changing  calibration  intervals 
for  equipment  it  controls:  A  section  of  the  Standard  specifies  the  requirements  for 
establishing  calibration  intervals.  These  procedures  should  be  included  in  the 
quality  documentation. 

21)  A  statement  of  the  laboratory's  policy  concerning  the  technique(s)  to  be  used  for 
determining  measurement  uncertainty  and  calibration/verification  adequacy:  A 
section  of  the  Standard  covers  the  requirements  for  determining  measurement 
uncertainty  and  calibration/verification  adequacy.  A  policy  statement  referencing 
these  procedures  should  be  included  in  the  quality  documentation. 

2.2.22  Excimple(s). 

2. 2. 2. 2.1  The  relation  between  management,  technical  operations,  support  services  and  the 
quality  system.  Flow  charts  or  written  descriptions  of  work  processes  defining  the 
responsibilities  of  and  demonstrating  interactions  between  the  various  organizational  units  might 
be  provided. 

2. 2. 2. 2. 2  Job  Descriptions  of  key  staff  and  reference  to  descriptions  of  other  staff.  Key 
personnel  might  be  defined  as  Manager(s),  Authorized  Representative(s),  Approved  signatory(s), 
quality  manager,  senior  metrologist,  senior  engineer,  and  others.  Other  staff  might  include 
engineers,  technicians,  administrative  support  personnel,  and  shipping  and  handling  personnel. 

2. 2. 2. 2. 3  The  laboratory's  scope  of  calibrations  and/or  verifications.  An  effective  method  of 
reporting  a  laboratory's  capabilities  is  in  a  spreadsheet  format.  Citing  a  basic  measurement 
discipline  (MWIR  or  LWIR)  and  then  the  various  sub-parameters  under  that  specific  discipline 
(radiometric,  spectral  responsivity,  emissivity,  etc.).  Other  information  provided  in  this 
spreadsheet  may  include  the  range,  accuracy  and/or  uncertainty,  technique  or  method  of 
calibration,  and  various  remarks  specific  to  the  sub  -parameter. 
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2. 2. 2. 2. 4  Reference  to  the  calibration  and  verification  procedures  used.  A  laboratory’s  quality 
manual  might  reference  calibration  procedures  as  follows:  “All  calibrations  conducted  by  this 
laboratory  are  performed  in  accordance  with  approved  calibration  procedures.  Development 
and/or  selection  of  procedures  are  based  on  suitability  for  purpose  and  customer  requirements. 
All  procedures  are  developed  or  selected  and  approved  for  use  in  accordance  with  guidelines 
found  in  Operational  Procedure  #XXX,  “Calibration  Procedure  Selection  and  Development”. 
Procedures  are  controlled  documents  with  the  designated  approval  signature(s)  and  issue  date  in 
accordance  with  Operational  Procedure  #ZZZ:  “Document  Control.” 

2. 2. 2. 2. 5  The  laboratory  management  arrangements  for  exceptionally  permitting  departures 
from  documented  policies  and  procedures  or  from  standard  specifications.  An  example  of  a 
statement  concerning  deviation  from  established  policies  and  the  quality  manual  might  be  as 
follows:  Any  deviations  from  the  guidelines  in  this  document  (quality  manual)  shall  be 
described  with  supporting  justification  and  shall  have  prior  approval  from  the  Calibration 
Laboratory  Senior  Management.  Individuals  desiring  deviation  from  the  requirements  set  forth 
in  this  document  (quality  manual)  shall  provide  to  the  Calibration  Metrology  Management  or 
designee  prior  to  implementation  a  detailed  description  and  justification  for  the  proposed 
deviations  for  review.  A  complete  description,  justification,  and  record  of  approval  shall  be 
documented  and  maintained  for  history. 

22.2.2.6  Procedures  for  protecting  confidentiality  and  proprietary  rights.  Details  of  these 
procedures  may  be  incorporated  into  the  procedures  for  reviewing  new  work  and/or  handling  of 
calibration  items. 


2.2.3  Quality  Audit.  The  laboratory  shall  arrange  for  audits  of  its  activities  at  appropriate 
intervals  to  verify  that  its  operations  continue  to  comply  with  the  requirements  of  the  quality 
system.  Such  audits  shall  be  carried  out  whenever  possible  by  trained  and  qualified  staff  who  are 
independent  of  the  activity  to  be  audited.  When  the  audit  findings  cast  doubt  on  the  correctness 
of  validity  of  the  laboratory's  calibrations  results,  the  laboratory  shall  take  immediate  corrective 
action  and  shall  immediately  notify,  in  writing,  any  client  whose  work  may  have  been  affected. 

2.2. 3.1  Interpretive  Guidance.  A  quality  audit  is  a  periodic  check  to  ensure  that  the 
laboratory  is  operating  in  accordance  with  the  policies  and  procedures  set  out  in  the  quality 
manual  and  related  documentation.  Quality  audits  may  include  audits  of  results.  Audit  intervals 
should  be  based  on  an  analysis  of  factors  such  as  risk,  criticality  of  the  measurement  system, 
complexity  of  the  system,  and  failure  rates  and  may  be  different  for  each  measurement  area. 
Audits  should  be  conducted  by  persons  possessing  the  knowledge  and  skills  necessary  to 
understand  the  process  being  audited  as  well  as  the  process  of  auditing.  Auditors  are  preferably 
persons  from  outside  the  area  being  audited.  Immediate  corrective  action  refers  to  the  speed  and 
breadth  of  response  necessary  to  prevent  the  taking  of  bad  data  and/or  the  generation  of 
inaccurate  results. 

2.2. 3.2  Example(s).  The  quality  manager  may  be  responsible  for  ensuring  that  all 
components  of  the  laboratory's  activities  are  audited.  The  task  of  carrying  out  audits  may  be 
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delegated  to  other  staff  with  sufficient  technical  knowledge  and  appropriate  auditing  training. 

The  audit  process  might  include: 

1)  A  planned  schedule  for  the  audits  covering  all  quality  activities  at  all  sites  over  a 
specified  period  of  time. 

2)  Audits  conducted  by  someone  independent  of  the  activity  concerned  (wherever 
possible). 

3)  Documented  audit  procedures. 

4)  Recording  of  audit  results. 

5)  Effective  corrective  action,  undertaken  within  a  reasonable  and  agreed  upon 
timeframe,  on  all  non-conformances  identified  within  the  quality  system. 

Each  aspect  of  the  quality  system  should  be  periodically  audited  at  an  appropriate  interval 
to  provide  sufficient  data  for  an  effective  management  review.  Many  organizations  find  that  an 
interval  of  six  or  twelve  months  to  be  appropriate. 

2.2.4  Quality  System  Review.  The  quality  system  adopted  to  satisfy  the  requirements  of 
this  Standard  shall  be  reviewed  at  least  once  a  year  by  the  management  to  ensure  its  continuing 
suitability  and  effectiveness  and  to  introduce  any  necessary  changes  or  improvements. 

2.2.4. 1  Interpretive  Guidance.  A  quality  system  review  is  an  examination  of  the  entire 
quality  system  conducted  by  management  to  ensure  that  it  is  adequate  for  the  achievement  of  the 
laboratory's  quality  objectives  on  a  continuous  basis. 

2. 2. 4. 2  Example(s).  The  quality  manager,  under  direction  of  laboratory  management,  might 
be  responsible  for  ensuring  that  all  reviews  are  done  systematically,  according  to  a  documented 
procedure,  that  results  of  revisions  are  recorded  and  that  all  actions  resulting  from  the  review  are 
implemented  within  the  specified  time.  The  review  could  include  at  least  the  following: 

1)  Matters  arising  from  the  previous  review. 

2)  Reports  from  audits  by  clients  (if  applicable). 

3)  Results  of  internal  audits  done  since  the  last  review,  including  corrective  actions 
implemented. 

4)  Results  of  in-house  quality  checks. 

5)  Details  of  any  complaints  from  clients. 

6)  Staff  training  (for  both  new  and  existing  staff  members). 

7)  Adequacy  of  staff,  equipment  and  facility  resources. 

8)  Future  plans  and  estimates  for  new  work,  new  staff,  new  equipment,  etc. 

Management  may  delegate  the  responsibility  of  scheduling  the  reviews  to  the  quality  manager  or 
other  appropriate  individuals. 

2.2.5  Documentation.  All  audit  and  review  findings  and  any  corrective  actions  that  arise 
from  them  shall  be  documented.  The  person  responsible  for  quality  shall  ensure  that  these 
actions  are  discharged  within  the  agreed  timeframe. 

2.2. 5.1  Interpretive  Guidance.  Documentation  should  be  maintained,  in  a  form  suitable  for 
assessment,  of  all  audits  and  management  reviews  of  the  quality  system  and  resulting  corrective 
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actions  (if  any).  Corrective  action  is  required  whenever  evidence  arises  that  the  quality  system  is 
not  functioning  properly.  The  person  responsible  for  quality  is  designated  by  laboratory 
management  and  is  normally  the  quality  manager. 

2.2.6  Quality  Check.  In  addition  to  periodic  audits,  the  laboratory  shall  ensure  the  quality 
of  results  provided  to  customers  by  implementing  checks.  These  checks  shall  be  reviewed  and 
shall  include,  as  appropriate,  but  not  be  limited  to  the  following: 

1)  Internal  quality  control  using,  wherever  possible,  statistical  techniques. 
Measurement  assurance  techniques  are  acceptable  means  to  control  the 
measurement  process  and  consistently  produce  the  highest  quality  measurements. 

2)  Participation  in  proficiency  testing  or  other  inter  laboratory  comparisons. 

3)  Regular  use  of  certified  reference  materials  and/or  in-house  quality  control  using 
reference  materials. 

4)  Replicate  measurements  using  the  same  or  different  methods. 

5)  Correlation  of  results  for  different  characteristics  of  an  item. 

2.2.6. 1  Interpretive  Guidance.  Internal  quality  control  programs  are  an  integral  part  of  any 
calibration  laboratory  program.  The  extent  and  detail  of  the  program  may  vary  based  on  the  size 
and  complexity  of  the  laboratory's  range  of  calibrations. 

2. 2. 6.2  Example(s).  Laboratories  maintaining  primary  standards  such  as  standard  or  items 
may  wish  to  develop  detailed  statistical  process  control  charts  for  the  items  utilizing  check 
standards.  For  laboratories  of  lesser  measurement  capabilities  (larger  uncertainties),  internal 
random  sampling  schemes  may  be  incorporated  whereby  random  items  of  instruments  whose 
calibrations  are  completed  are  recalibrated  by  separate  laboratory  personnel.  These  sampling 
schemes  may  be  based  on  a  percentage  of  instruments  calibrated  where  the  percentage  may  be 
increased  or  decreased  based  on  the  number  of  failed  inspections. 

Internal  quality  control  methods  such  as  these  are  a  few  of  the  methods  that  may  be 
used  to  provide  statistical  data  supporting  the  credibility  of  the  calibration  laboratory 
"production"  and  the  reliability  and  stability  of  laboratory  standards. 

When  internal  quality  assurance  and  quality  control  techniques  are  not  implemented, 
decreases  in  laboratory  calibration  quality  may  not  be  identified  until  long  after  detrimental 
effects  to  quality  have  occurred.  This  could  cast  significant  doubt  on  the  measurement  system, 
the  validity  of  calibrations  performed  by  the  laboratory  and  all  measurements  and  measurement 
data  collected  utilizing  the  calibrated  equipment. 
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2.3  Personnel 

2.3.1  Staff  Size.  The  calibration  laboratory  shall  have  sufficient  personnel  with  the  necessary 
education,  training,  technical  knowledge  and  experience  for  their  assigned  functions. 

2. 3. 1.1  Interpretive  Guidance.  The  laboratory  should  maintain  a  staff  of  sufficient  size  so  as 
not  to  cause  unnecessary  delays  in  the  service  it  provides  to  its  customers.  Staff  members  should 
possess,  at  a  minimum,  the  training  and  skills  necessary  to  perform  their  assigned  duties. 

Training  should  be  “fit  for  purpose”. 

2. 3. 1.2  Example(s).  Several  schemes  are  available  for  monitoring  the  timeliness  of  the 
service  provided  by  the  laboratory,  usually  taking  the  form  of  a  workload,  or  backlog,  tracking 
system.  Data  collected  from  these  types  of  systems  may  be  used  to  evaluate  whether  or  not  the 
laboratory  is  providing  the  “speed”  of  service  to  its  customers  sufficient  to  satisfy  their  needs. 
Any  such  system  employed  need  only  be  as  complex  as  deemed  necessary  by  laboratory 
management  to  suit  its  particular  needs,  so  long  as  enough  information  is  collected  on  which  to 
base  a  determination.  Training  requirements  might  be  established  by  first  identifying  the  duties 
of  each  staff  member,  either  individually  or  by  job  classification  (supported  by  written  job 
descriptions),  and  then  determining  the  training  and/or  level  of  expertise  required  to  perform 
these  duties.  Training  should  be,  at  a  minimum,  “fit  for  purpose”.  For  example,  if  a  technician 
is  responsible  for  performing  a  statistical  analysis  of  data  collected  during  the  calibration 
process,  then  he/she  should  possess  the  necessary  skills  to  perform  such  an  evaluation.  These 
skills  may  be  obtained  from  a  variety  of  sources,  such  as  formal  education,  formal  training,  on 
the  job  training,  or  any  other  suitable  means.  Some  laboratories  may  also  require  their  staff  to 
demonstrate  their  competency  by  a  variety  of  methods  such  as  qualification  testing,  performance 
testing,  ongoing  proficiency  audits,  or  review  of  results  of  laboratory  proficiency  tests. 

2.3.2  Staff  Training.  The  calibration  laboratory  shall  ensure  that  the  training  of  its 
personnel  is  kept  up-to-date  consistent  with  employee  assignments  and  development. 

2. 3.2.1  Interpretive  Guidance.  The  laboratory  should  have  procedures  for  updating  and/or 
augmenting  the  training  of  its  staff  as  requirements  change.  These  procedures  should  include 
provisions  for  recognizing  when  changes  that  may  require  training  occur  and  how  that  training 
might  be  accomplished. 

2. 3.2.2  Example(s).  Changes  may  occur  for  a  variety  of  reasons,  such  as,  but  not  limited  to, 
reassignment  of  duties,  new  work  taken  on  by  the  laboratory,  changes  in  technology,  updated 
methods  and  procedures,  cross  training  of  personnel  and  employee  development.  Job 
descriptions  and  associated  training  requirements  might  be  periodically  reviewed  and  modified 
as  necessary  to  maintain  compatibility  with  current  measurement  technology  and  to  remain 
consistent  with  the  laboratory’s  mission. 

2.3.3  Staff  Records.  Records  on  the  relevant  qualifications,  training,  skills  and  experience 
of  the  technical  personnel  shall  be  maintained  and  be  available  to  the  laboratory. 

2. 3. 3.1  Interpretive  Guidance.  A  current  listing  of  technical  personnel  and  their  relevant 

skills,  work  experience,  formal  education,  technical  training,  on-the-job  training,  technical 
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license,  certifications,  and  degrees  should  be  maintained.  All  such  information  should  be 
available  to  laboratory  management. 

233.2  Example(s).  The  documentation  may  be  a  controlled  computerized  list  generated  by 
the  training  department,  personnel  office,  laboratory  supervisor  or,  simply  a  collection  of 
documents  contained  in  the  employees  personnel  file.  The  listing  should  be  periodically 
reviewed  by  the  employee  and  laboratory  management  to  ensure  accuracy  and  completeness. 

2.4  Accommodation  and  Environment 

2.4.1  Laboratory  Environmental  Conditions.  Laboratory  accommodation  (facilities), 
calibration  areas,  energy  sources,  lighting,  temperature,  humidity,  and  ventilation  shall  be  such 
as  to  facilitate  proper  performance  of  calibrations/verifications. 

2. 4. 1.1  Interpretative  Guidance.  Environmental  conditions  should  be  appropriate  for  the 
type  of  work  being  performed.  Calibration/verification  requirements  should  be  evaluated  to 
determine  which  environmental  factors  have  an  impact.  Only  the  environmental  factors  which 
could  affect  the  accuracy,  stability,  or  performance  of  the  calibration  require  control.  Human 
factors  such  as  lighting  and  ventilation  should  be  sufficient  to  perform  the  required  tasks. 

2. 4. 1.2  Example(s).  The  temperature  control  requirements  for  the  calibration  of  an  infrared 
detector  would  be  tighter  than  the  requirements  for  the  calibration  of  other  general-purpose 
equipment,  which  has  a  wide  temperature  operating  range. 

2.4.2  Experiment  Environmental  Conditions.  The  environment  in  which  these  activities  are 
undertaken  shall  be  specified  and  not  invalidate  the  results  or  adversely  affect  the  required 
uncertainty  of  measurement.  Particular  care  shall  be  taken  when  such  activities  are  undertaken  at 
sites  other  than  the  permanent  laboratory  premises. 

2.4.2. 1  Interpretative  Guidance.  Environmental  influence  factors  which  affect  the  specific 

measurement  being  performed  should  be  understood,  documented  and  controlled  to  the  degree 
necessary  to  avoid  invalidating  the  calibration  results  or  adversely  affecting  the  required 
measurement  uncertainty.  These  requirements  also  apply  to  off-site  calibration  in  that  care 
should  be  taken  to  monitor,  record,  and  compensate  for  environmental  factors  which  could 
adversely  affect  the  measurement  and/or  its  uncertainty.  In  an  environmentally  controlled  area, 
an  out  of  tolerance  environmental  parameter  may  not  affect  the  required  uncertainty. 

Correction  factors  may  be  applied  to  compensate  for  the  out  of  tolerance  condition,  thus 
allowing  some  work  to  continue. 
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2.4.3  Monitor,  Control,  and  Record.  The  laboratory  shall  effectively  monitor,  control  and 
record  environmental  conditions  as  appropriate.  Due  attention  shall  be  paid,  for  example,  to 
biological  sterility,  dust,  electromagnetic  interference,  humidity,  line  voltage,  temperature,  and 
sound  and  vibration  levels,  as  appropriate  to  the  calibrations  concerned. 

2. 4. 3.1  Excimple(s).  In  an  infrared  calibration  laboratory,  temperature  is  critical  and  should 
be  monitored,  controlled,  and  recorded.  Dust  levels  are  also  critical  when  making  optical 
measurements.  Humidity  should  be  controlled  so  as  to  prevent  corrosion  of  standards. 

2.4.4  Sub  Zoning.  There  shall  be  effective  separation  between  neighboring  areas  when  the 
activities  therein  are  incompatible. 

2.4.4. 1  Interpretative  Guidance.  Activities  in  neighboring  areas  may  have  an  adverse  affect 
on  the  calibrations/verifications  in  another  area.  If  activities  are  incompatible,  they  should  be 
effectively  separated  by  shielding  or  isolated  by  distance. 

2. 4. 4. 2  Example(s).  Line  voltage  surges  may  be  caused  by  motors  or  other  high  current 
drawing  industrial  equipment  in  neighboring  areas.  Line  conditioners  or  regulated  power 
sources  should  be  employed  to  protect  sensitive  electronic  equipment. 

2.4.5  Access  Control.  Access  to  and  use  of  all  areas  affecting  the  quality  of  these  activities 
shall  be  defined  and  controlled. 

2. 4. 5.1  Interpretative  Guidance.  Uncontrolled  access  to  environmentally  controlled  areas 
may  have  an  adverse  affect  on  the  quality  of  the  measurements  being  performed.  Areas  where 
this  potential  exists  should  be  identified  and  procedures  should  be  established  to  control  access. 

2.4. 5.2  Example(s).  Unqualified  personnel  could  unknowingly  cause  damage  to  calibration 
equipment  or  adjust  calibration  settings.  Access  to  areas  could  be  controlled  by  physical  locks, 
security  guards  or  other  security  access  systems. 

The  number  of  personnel  entering  an  environmentally  controlled  area  should  be 
limited.  The  more  personnel  entering  the  area  the  more  dust  and  dirt  will  be  introduced.  If  too 
many  people  are  in  the  laboratory,  their  body  heat  could  raise  the  temperature  to  an  out-of¬ 
tolerance  level. 

Air  lock  systems  can  be  used  to  help  minimize  the  effects  on  temperature  control 
when  personnel  enter  an  environmentally  controlled  area. 

Electrostatic  discharge  workstations  and  special  procedures  should  be  used  when 
working  with  some  sensitive  electronic  equipment. 
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2.4.6  Housekeeping.  Adequate  measures  shall  be  taken  to  ensure  good  housekeeping  in  the 
laboratory.  It  is  the  laboratory's  responsibility  to  comply  with  the  relevant  health,  safety,  and 
environmental  requirements.  This  aspect,  however,  is  outside  the  scope  of  this  Standard. 

2. 4. 6.1  Interpretative  Guidance.  Poor  housekeeping  practices  may  have  a  negative  impact  on 
the  quality  of  the  calibrations/verifications  being  performed.  Good  housekeeping  factors  such  as 
cleanliness,  storage  and  space  should  be  adequately  controlled. 

2. 4. 6.2  Example(s).  Improper  storage  could  cause  damage  to  or  degrade  the  accuracy  of 
standards  and  measuring  equipment.  Some  examples  of  improper  storage  are: 

1)  Stacking  equipment  too  high. 

2)  Outside  storage  exposing  equipment  to  the  elements. 

3)  Storage  in  areas  which  exceed  storage  specifications. 

4)  Equipment  not  properly  stored  or  secured  after  use. 

Cleanliness  of  work  areas  and  calibration  equipment  is  critical  when  making 
precision  measurements.  A  particle  of  dust  or  dirt  could  invalidate  the  calibration  results. 

Improper  document  storage  could  result  in  the  loss  of  valuable  quality  records. 

Record  files  should  be  properly  maintained  and  computer  disks  stored  away  from 
electromagnetic  sources. 

Filters  on  optical  measurement  equipment  should  be  kept  clean. 

Improper  storage  of  reference  materials  could  affect  their  reliability  or  accuracy. 

Improper  maintenance  of  lighting  fixtures,  such  as  failure  to  replace  light  bulbs,  could 
cause  lighting  levels  to  drop  to  unacceptable  levels. 


7-20 


RCC  Special  Report:  Development  Of  Electro-Optical  (EO)  Standard  Processes  for  Application,  November  2011 


2.5  Equipment  and  Reference  Materials 

2.5.1  Equipment  Availability.  The  laboratory  shall  be  furnished  with  all  items  of  equipment 
(including  reference  materials)  required  for  the  correct  performance  of  calibrations/verifications. 
In  those  cases  where  the  laboratory  needs  to  use  equipment  outside  its  permanent  control,  it  shall 
ensure  that  the  relevant  requirements  of  this  Standard  are  met. 

2. 5. 1.1  Interpretive  Guidance.  Laboratories  should  have  available  all  equipment  needed  for 
the  calibrations  they  normally  perform,  whether  owned,  leased,  rented,  or  borrowed.  The 
calibration  laboratory  should  assure  that  leased,  rented  or  borrowed  equipment  meets  the 
requirements  of  this  Standard.  Noncompliance  with  this  requirement  could  lead  to  partial 
calibrations,  or  calibrations  using  equipment  of  unknown  integrity. 

2. 5. 1.2  Example(s).  Calibration  laboratory  "A"  must  rent  a  spectrometer  in  order  to  complete 
a  calibration.  "A"  does  not  have  the  capability  to  verify  the  performance  of  the  spectrometer  and 
must  rely  on  the  rental  company's  calibration  certification/sticker.  Before  using  the 
spectrometer,  "A"  should  have  on  file  or  obtain  objective  evidence  that  the  calibration  was 
performed  in  a  manner  which  meets  the  requirements  of  this  Standard. 

Examples  of  objective  evidence  may  be: 

1)  Records  of  your  own  audits. 

2)  Proof  that  the  rental  company’s  calibration  laboratory  is  accredited  against  this 
Standard. 

3)  Acceptance  of  the  rental  company’s  calibrations  by  a  government  agency  showing 
compliance  to  this  Standard. 

4)  Acceptance  conforming  to  this  Standard  from  the  calibration  laboratory’s 
customer. 

2.5.2  Equipment  Maintenance.  All  equipment  shall  be  properly  maintained.  Maintenance 
procedures  shall  be  documented.  Any  item  of  the  equipment  which  has  been  subjected  to 
overloading  or  mishandling,  or  which  gives  suspect  results,  or  has  been  shown  by  verification  or 
otherwise  to  be  defective,  shall  be  taken  out  of  service,  clearly  identified  and  wherever  possible 
stored  at  a  specified  place  until  it  has  been  repaired  and  shown  by  calibration  or  verification  to 
perform  satisfactorily.  The  laboratory  shall  examine  the  effect  of  this  defect  on  previous 
calibrations. 

2. 5.2.1  Interpretive  Guidance.  Maintenance  of  a  calibration  laboratory's  equipment  should 
be  controlled.  A  calibration  laboratory  should  have  procedures  describing  how  it  maintains  and 
controls  equipment  used  to  perform  calibrations  under  this  Standard.  These  procedures  should 
state  in  detail: 

1)  How  historical  maintenance  information  is  kept. 

2)  How  an  instrument  that  has  been  subjected  to  any  influence  that  might  cause 
doubt  as  to  its  integrity  is  handled. 

3)  How  out  of  service  equipment  is  marked. 

4)  How  effects  on  previous  calibrations  are  determined. 

5)  How  operational  status  is  identified. 

6)  Specifically  where  equipment  is  to  be  kept  while  out  of  service. 
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Failure  to  comply  with  this  requirement  could  cause  a  technician  to  use  defective 
equipment  unknowingly  in  the  process  of  calibration. 

2.5.3  Equipment  Labels.  Each  item  of  equipment  including  reference  material  shall,  when 
appropriate,  be  labeled,  marked,  or  otherwise  identified  to  indicate  its  calibration  status. 

2. 5. 3.1  Interpretive  Guidance.  Any  item  that  may  be  used  in  the  calibration  process  should 
have  evidence  of  its  calibration  and  service  status.  If  an  item  can  be  or  might  be  used  as  a  test 
standard,  it  should  be  marked  such  that  a  user  can  easily  determine  its  calibration  status.  This 
prevents  use  of  an  item  which  may  not  be  calibrated,  may  be  defective,  or  may  have  an  expired 
calibration  date.  Reference  materials  should  be  treated  in  a  manner  similar  to  equipment  and 
have  their  current  status  identified  in  a  manner  that  will  allow  a  user  to  determine  their  validity. 
Typical  identification  of  status  is  through  the  use  of  calibration  labels,  indicating  the  type  of 
calibration  (limited,  special,  see  report  etc.)  and  calibration  due  date.  However,  other  schemes, 
such  as  computer  systems  keyed  on  instrument  serial  numbers,  would  be  acceptable  as  long  as 
the  process  is  well  documented  and  understood  by  all  potential  users. 

2. 5. 3.2  Excimple(s). 

1)  "Calibration  Labels"  should  be  applied  to  equipment  when  its  accuracy 
significantly  affects  the  integrity  of  the  measurement  process. 

2)  If  calibration  labels  cannot  be  applied  (gage  blocks,  weights,  small  items,  etc.)  the 
information  may  be  placed  on  the  equipment  container. 

3)  Local  policies  and  procedures  should  clearly  instruct  all  equipment  users  that  an 
instrument  is  only  considered  useable  for  a  calibration  if  a  valid  and  current 
“label”  (or  other  acceptable  means  of  identification)  is  available. 

2.5.4  Equipment  Records.  Records  shall  be  maintained  of  each  item  of  equipment  and  all 
reference  materials  significant  to  the  calibrations/verifications  performed.  The  records  shall 
include: 

a.  The  name  of  the  item  of  equipment. 

b.  The  manufacturer's  name,  type  identification,  and  serial  number  or  other  unique 
identification. 

c.  Current  location,  where  appropriate. 

d.  Where  applicable,  dates  and  results  of  calibration  and/or  verifications  and  date  or 
criteria  when  the  calibration  and/or  verification  expires. 

e.  Details  of  maintenance  carried  out  to  date  and  planned  for  the  future. 

f.  History  of  any  damage,  malfunction,  modification  or  repair. 

g.  Measured  value  observed  for  each  parameter  found  to  be  out  of  tolerance  during 
calibration/verification. 

2. 5. 4.1  Interpretive  Guidance.  The  laboratory  should  keep  historical  files  of  calibrations  of 
all  laboratory  equipment  significant  to  the  calibration  process,  containing  sufficient  information 
to  prove  calibration  credibility  at  the  time  of  each  use.  This  includes  documented  measured 
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value  data  of  any  parameter  found  to  be  out  of  tolerance  during  initial  verification,  but  does  not 
require  that  data  be  kept  on  all  parameters  measured  and  found  to  be  in  tolerance. 

2. 5. 4. 2  Example(s). 

1)  Model:  Model  xxx. 

2)  S/N:  xxx-xx-xx. 

3)  Name/Type:  Blackbody. 

4)  Manufacturer:  Any  Corporation. 

5)  Location:  3975  5th  Street. 

6)  Last  Cal:  November  17,  2006  Condition:  REPAIR  required. 

7)  Next  Cal:  November  17,  2007. 

8)  Maintenance  History:  1/20/2004  received  new. 

9)  1 1/17/93  Temperature  control  failed;  repaired  and  installed  reliability  upgrades; 
calibrated. 

10)  Reading  history:  (See  calibration  certificate/report). 

2.6  Measurement  Traceability  and  Calibration 

2.6.1  Calibration  System.  All  measuring/testing  equipment  having  an  effect  on  the  accuracy  or 
validity  of  calibrations  shall  be  calibrated  and/or  verified  before  being  put  into  service.  The 
laboratory  shall  have  an  established  program  for  the  calibration  and  verification  of  its  measuring 
and  test  equipment  to  ensure  the  recall  or  removal  from  service  of  any  standard  or  equipment 
which  has  exceeded  its  calibration  interval  or  is  otherwise  judged  to  be  unreliable. 

2.6. 1.1  Interpretive  Guidance.  A  calibration  laboratory  should  have  a  calibration  system  for 
all  equipment  utilized  in  the  calibration  laboratory.  This  system  should  allow  for: 

1)  Identification  of  equipment  requiring  calibration. 

2)  Identification  of  overdue  items  in  the  recall  system  and  removal  of  such  items 
until  corrected. 

3)  Items  that  are  known  to  be  broken,  out  of  tolerance,  unstable  so  that  its  parameters 
cannot  be  predicted,  or  have  intermittent  problems,  etc.,  or  are  judged  unreliable, 
are  identified  and  segregated  until  appropriate  corrective  action  has  been  taken. 

Corrective  action  may  be  in  the  form  of  limited  use,  repair,  or  correction  factors,  etc. 

2.6. 1.2  Example(s).  There  are  usually  two  distinct  groups  of  instruments  involved  in  any 
calibration  system: 

1)  Those  that  belong  to  or  are  assigned  to  the  calibration  laboratory  itself.  The 
calibration  laboratory  should  keep  tight  control  of  all  its  equipment  used  for 
calibration.  This  includes  periodic  recalibration  of  its  instruments/tools  in  most 
cases.  Exceptions  should  be  listed  and  identified  as  such. 

2)  Those  that  belong  to  or  are  assigned  to  customers  of  the  calibration  laboratory  or 
that  are  maintained  through  an  instrument  pool  or  lease/rental  program.  Control 
of  these  instruments/tools  is  usually  left  to  the  user  or  its  assigned  delegate.  It  is 
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up  to  the  user  to  identify  items  requiring  periodic  recalibration,  to  provide 
adequate  identification,  and  maintain  a  tracking  program  for  his  equipment. 

The  calibration  laboratory  should  have  complete  control  of  its  inventory  of 
instruments  and  tools  used  for  the  calibration  and  verification  of  its  own  items.  These 
instruments/tools,  utilized  by  the  calibration  laboratory,  could  be  owned,  borrowed,  leased,  or 
rented  from  other  facilities  or  signed  out  from  loan  pools.  In  all  cases,  those  items  requiring 
calibration  or  verification  should  be  fully  accounted  for  at  all  times  and  should  be  in  a 
calibrated/verified  status  through  the  use  of  an  official  calibration  recall  system. 

It  is  important  to  note  that  not  ALL  instruments/tools  used  by  the  calibration 
laboratory  for  calibration  or  verification  do  require  such  actions.  The  calibration  laboratory 
decides  which  items  are  used  as  sources  or  indicators,  thereby  requiring  no  calibration,  and 
which  items  have  an  effect  on  the  accuracy  for  the  units  under  test. 

A  recall  system  will  mean  different  things  for  different  organizations,  facilities,  or 
agencies.  In  general,  it  can  be  said,  however,  that  a  recall  system  should  include: 

1)  Identification  of  items  requiring  calibration  or  verification. 

2)  A  detailed  database  for  items  requiring  ca  libration  or  verification  that  indicates 
past  and  future  calibration  dates,  etc. 

3)  A  description  of  how  the  instruments/tools  requiring  calibration  and  verification 
are  moved  to  and  from  different  calibration  laboratories,  how  they  are  calibrated 
or  verified  including  what  procedures  are  used,  and  how  calibration  records  are 
kept. 

The  recall  system,  through  its  database,  should  provide  a  means  for  identifying 
calibration  laboratory  items  that  are  overdue  for  calibration.  The  calibration  laboratory  should 
ensure  that  such  items  are  taken  out  of  service  and  segregated  until  recalibrated. 

Problem  items  such  as  repeated  out-of-tolerance  conditions  of  the  same  parameter  of 
the  same  unit  should  be  investigated  for  cause  and  possible  problems  downstream.  Appropriate 
action  should  be  taken  by  the  laboratory  to  repair,  derate,  or  eliminate  such  units  from  the 
inventory.  A  mechanism  should  be  available  to  identify  these  items.  Items  that  show  abnormal 
behavior  should  be  temporarily  removed  from  the  active  inventory  until  the  problems  have  been 
clearly  identified  and  can  be  controlled. 

Items  that  are  broken  should  be  removed  from  the  active  inventory  and  identified  as 
such.  Items  for  which  trend  and  control  charts  are  kept  and  which  indicate  that  certain 
parameters  are  falling  outside  the  predicted  control  limits,  should  be  removed  from  the  active 
inventory  list,  identified  as  such,  and  investigated  for  cause.  Once  understood  and  corrected  they 
can  be  entered  into  the  active  inventory  once  again  after  adequate  calibration. 

2.6.2  Calibration  Traceability.  The  overall  program  of  calibration  and/or  verification  of 
equipment  shall  be  designed  and  operated  so  as  to  ensure  that,  wherever  applicable, 
measurements  made  by  the  laboratory  are  traceable  to  national,  international,  or  intrinsic 
standards  of  measurements  where  available.  Calibration  certificate  and/or  report  shall,  wherever 
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applicable,  state  the  traceability  to  national,  international,  or  intrinsic  standards  of  measurements 
and  shall  provide  the  measurement  results  and  associated  uncertainty  of  measurement  and/or  a 
statement  of  compliance  with  an  identified  metrological  specification. 

2.6.2. 1  Interpretive  Guidance.  In  order  to  ensure  adequate  calibration  quality,  all  calibration 
laboratory  items  in  the  recall  system  should  be  traceable  to  national  or  international  laboratories 
or  to  intrinsic  standards,  where  available.  Where  applicable,  a  specific  traceability  statement 
should  be  provided  in  the  calibration  certificate  or  report  identifying  the  source  of  traceability, 
i.e.,  national  or  international  laboratory  or  intrinsic  standard.  It  is  not  necessary  to  state  further 
identifiers  such  as  bookkeeping  numbers,  NIST  report  numbers,  etc.  Stated  numeric 
measurement  results  on  calibration  certificates  or  reports  should  include  the  associated 
uncertainties.  For  further  background  on  the  concept  of  traceability,  and  its  evolution  in  defining 
the  pathway  to  International  System  of  Units  (SI). 

2. 6. 2. 2  Example(s).  All  calibrations  and/or  verifications  of  the  calibration  laboratory 
equipment  should  be  traceable  to  one  ultimate  source.  However,  not  all  equipment  used  in  the 
laboratory  requires  traceability.  For  example: 

1)  Equipment  used  in  an  indicative  mode  or  used  for  troubleshooting  and  repair.  In 
these  cases,  absolute  reference  to  an  ultimate  source  is  not  necessary  and  may  be 
an  economic  burden.  Such  equipment  should  not  be  in  a  recall  system  for 
calibration. 

2)  Equipment  used  as  sources  with  other  traceable  instruments  employed  as 
measuring  units  for  the  output  of  these  sources. 

3)  Those  items  remaining,  in  most  instances,  exhibit  drifts  and  hence  do  require 
calibration  and/or  verification  and  should  be  traceable  where  available.  The  key 
here  is  "where  available".  It  is  quite  possible  that  a  certain  measurement  should 
be  traceable  but  no  ultimate  standards  or  artifacts  exist.  In  some  cases,  consensus 
standards  are  developed  and  used  until  (if  ever)  such  times  when  official  national 
or  international  standards  have  been  approved  for  general  use.  It  is  important  to 
note  that  "where  available"  should  not  be  misused.  The  metrologist  has  to  be  sure 
that  traceability  is  not  possible  in  the  national  and  international  arena.  The  only 
way  to  know  for  sure  is  by  networking  with  colleagues  in  related  industries  and 
personnel  from  the  national  and  international  laboratories.  However,  once  a  piece 
of  equipment  has  been  classified  as  needing  traceability  and  there  is  a  means  of 
obtaining  traceability,  the  calibration  laboratory  has  several  options  shown  below. 

a.  Traceability  through  a  sister  organization.  In  this  case,  the  traceable 
equipment  is  calibrated  or  verified  at  a  related  laboratory  within  the 
organization.  This  laboratory  then  has  to  be  able  to  provide  traceability  to 
national,  international,  or  intrinsic  standards. 

b.  Traceability  through  another  commercial  calibration  laboratory.  Once  again, 
this  commercial  calibration  laboratory  has  to  be  able  to  provide  proof  that  the 
specific  measurement  required  is  traceable  to  national,  international,  or 
intrinsic  standards. 

c.  Traceability  to  national  laboratories.  In  most  cases  this  is  the  easiest,  but  also 
the  most  expensive,  way  to  provide  traceability.  The  National  Standards 
Laboratory  in  the  respective  country  should  provide  the  ultimate  traceable 
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measurement  of  a  particular  parameter.  Just  by  making  the  measurement, 
stating  its  uncertainty,  and  providing  an  adequate  calibration  report  with  a 
specific  date,  traceability  is  guaranteed;  there  is  no  need  to  quote  report 
numbers  and  any  other  non-value-added  bookkeeping  numbers.  This  report 
by  itself  is  “traceability.” 

d.  Traceability  to  international  laboratories.  Increasingly,  a  calibration 
laboratory  will  find  that  one  particular  National  Standards  Laboratory  cannot 
provide  traceability  for  all  parameters,  or  that  certain  standards  are  only 
available  in  certain  countries  and  hence  measurement  traceability  has  to  be 
sought  through  those  countries.  This  will  become  more  apparent  as  the 
metrology  community  organizes  itself  into  regions  such  as  European,  North 
America,  and  the  Pacific  Rim.  Within  each  region,  agreements  might  be 
reached  that  certain  standards  are  maintained  in  certain  countries  and  then  use 
mutual  agreements  to  recognize  each  other’s  standards. 

e.  Traceability  to  intrinsic  standards  which  are  physically  stand-alone  standards 
with  a  high  degree  of  reproducibility.  Performance  wise,  they  have  to  be  tied 
into  intercomparison  programs  to  avoid  the  possibility  of  system  errors.  With 
regard  to  types  of  intrinsic  standards,  several  possibilities  exist  and  the  list  of 
such  standards  is  growing  every  year.  Some  examples  are  listed  here: 

•  Fundamental  constants  of  nature. 

•  Invariant  properties  of  nature. 

2.6.3  Alternatives.  Where  traceability  to  international,  national,  or  intrinsic  standards  of 
measurement  is  not  available,  traceability  requirements  may  be  satisfied  by: 

a.  Participation  in  a  suitable  program  of  interlaboratory  comparisons  or  proficiency 
testing. 

b.  Internationally  accepted  standards  in  the  field  concerned. 

c.  Suitable  reference  materials. 

d.  Ratio  or  reciprocity-type  measurements. 

e.  Mutual  consent  standards  which  are  clearly  specified  and  mutually  agreed  upon 
by  all  parties  concerned. 

2. 6. 3.1  Interpretive  Guidance.  In  addition  to  traceability  to  national  or  international 
standards  laboratories,  the  above  items  0  through  0  can  be  used  to  fulfill  traceability  arguments. 

2.6. 3.2  Example(s).  1)  Interlaboratory  comparisons  or  proficiency  testing.  Industrial  need 

for  primary,  traceable  standards  has  put,  in  many  cases,  an  overwhelming  burden 
on  the  National  Standards  Laboratories  (NSLs).  To  relieve  the  workload  of  these 
NSLs,  interlaboratory  comparisons  have  been  established.  These  programs  are 
also  known  as  Measurement  Assurance  Programs  (MAPs)  or  Round  Robins 
(RRs).  In  cases  for  which  official  traceability  to  national  standards  is  required, 
the  participation  of  an  NSL  or  an  accredited  laboratory  (AL)  is  required  to  serve 
as  the  pivot  laboratory,  the  overseer  of  the  intercomparison.  (Other  MAPs  or  RRs 
without  the  participation  of  NSLs  or  ALs  are  very  useful  but  do  not  carry  the 
same  weight  in  terms  of  traceability.)  Many  examples  can  be  quoted  for  MAP  or 
RR  candidates:  DC  voltage,  DC  resistance,  Capacitance,  Fiber  Optic,  Power,  UV 
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Sources,  etc.  In  addition,  intercomparison  programs  between  NSLs  are  very 
common  and  are  used  to  provide  international  definitions  of  particular  parameters. 
Once  agreed  upon  by  the  participating  NSLs,  these  parameters  can  then  be 
facilitated  within  a  particular  country  through  direct  standards  measurements  or 
additional  intercomparison  programs.  Examples  would  include  AC  voltage 
standards,  spectral  irradiance,  etc.  Proficiency  testing  may  be  required  for 
accreditation.  Any  MAP  or  RR  should  help  satisfy  this  requirement.  In  addition, 
measurements  done  one  on  one  for  a  particular  standard  may  constitute  a 
proficiency  test  as  long  as  either  an  NSL  or  an  AL  is  one  of  the  participating 
parties. 

2)  Internationally  accepted  standards.  Several  different  types  of  standards  fall  into 
this  category: 

a.  Standards  that  are  accepted  internationally  by  all  concerned.  For  example, 
mass  with  the  unit  of  kg.  (The  only  standard  still  maintained  internationally  as 
an  artifact.) 

b.  Standards  that  are  agreed  upon  by  the  participating  National  Standards 
Laboratories  (NSLs). 

c.  Standards  that  are  accepted  internationally  but  not  available  in  a  particular 
country.  In  these  cases,  traceability  by  a  calibration  laboratory  has  to  be 
sought  on  the  international  level  without  the  involvement  of  its  own  national 
standardizing  laboratory.  For  example,  triple  Point  of  Gases  Cells  for 
temperature. 

3)  Suitable  reference  materials.  If  no  particular  artifacts  can  be  constructed  or  if 
artifacts  cannot  be  maintained  consistently,  the  traceability  can  be  through 
reference  materials.  For  example,  thermal  conductivity  of  graphite  and  metals, 
specular  spectral  reflectance  (plate  form),  electrical  resistivity  and  conductivity  of 
metals,  etc.  Other  Standard  Reference  Materials  (SRMs)  are  used  for  the 
construction  of  artifacts  or  standards.  Examples  would  be  pure  metals  with  four 
or  five  nines  (99.999%  pure).  These  SRMs,  as  utilized  in  standards,  require 
additional  approval  in  their  implementation.  Further  examples  include  solution 
calorimetry,  optical  microscopy  line  width  measurement  (wafer  form), 
photographic  step  tablets,  etc. 

4)  Ratio  or  reciprocity-type  measurements.  Every  calibration  laboratory  eventually 
gets  involved  in  ratiometric  measurements.  The  ratio  as  such  cannot  be  made 
traceable.  However,  other  parameters  needed  in  the  operation  of  these  ratio 
devices  should  be  traceable.  In  terms  of  reciprocity  measurements,  usually  three 
devices  of  the  same  type  are  compared  by  ratio  techniques.  All  possible  ratio 
combinations  are  required  and  detailed  calculations  should  be  made  to  assure 
consistency  of  the  measurements.  For  example,  in  the  case  of  three  devices,  ratios 
1:2,  1:3,  2:3  should  be  made.  The  results  of  these  three  intercomparisons  may 
then  be  used  to  establish  traceability. 

5)  Mutual  consent  standards.  Every  now  and  then,  a  situation  occurs  in  a  particular 
calibration  laboratory  for  which  no  national,  international,  or  intrinsic  standard 
exists.  In  this  case,  all  parties  involved  with  the  development  and  the  use  of  that 
technology  should  coordinate  their  efforts  and  establish  a  Mutual  Consent 
Standard.  (It  helps  to  have  the  NSL  involved  from  the  beginning.)  Such  mutual 
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consent  standards  require  detailed  intercomparison  programs  between  the 
participants,  a  designated  but  arbitrary  pivot  laboratory,  and  constant  vigilance. 

In  most  cases,  the  NSL  in  the  particular  country  will  eventually  find  a  way  to 
provide  national  or  international  traceability  and  standards.  Once  these  have 
become  available,  the  Mutual  Consent  Standards  should  be  adjusted,  if  necessary. 

2.6.4  Reference  Standards  Usage.  Reference  standards  of  measurements  held  by  the 
laboratory  shall  be  used  for  calibration  or  verification  only  and  for  no  other  purpose,  unless  it  can 
be  demonstrated  that  their  performance  as  reference  standards  has  not  been  invalidated. 

2.6.4. 1  Interpretive  Guidance.  Generally,  a  calibration  laboratory  employs  reference  and 
working  standards  with  the  former  providing  traceability  to  a  particular  source  and  the  latter 
being  used  for  calibration  of  equipment  have  to  be  employed  with  increased  uncertainties.  On 
the  other  hand,  some  laboratories  use  a  single  level  of  standards  to  accomplish  both  purposes.  In 
this  situation,  great  care  should  be  taken  to  avoid  degradation  of  the  single  level  of  standards.  In 
addition,  some  means  of  checking  them  for  damage,  wear,  and/or  erratic  behavior  should  be 
considered.  Although  the  Standard  does  allow  for  a  single  level  of  standards  within  a  laboratory, 
it  specifically  disallows  for  reference  standards  to  be  used  for  purposes  other  than  calibration  or 
verification  unless  it  can  be  demonstrated  that  the  performance  of  the  reference  standard  has  not 
been  invalidated.  Therefore,  using  the  same  item  for  frequent  and  routine  measurements  as  well 
as  a  reference  standard  would  be  inappropriate  unless  special  checks  and  precautions  were  made. 

2.6.5  Reference  Standards  Calibration.  Reference  standards  of  measurement  shall  be 
calibrated  by  a  competent  body  that  can  provide  traceability.  There  shall  be  a  program  of 
calibration  and  verification  for  reference  standards. 

2. 6. 5.1  Any  reference  standards  employed  should  be  traceable  to  a  competent  source.  It  is 
to  be  understood  that  competent  implies  ability  to  prove  adequacy  of  the  calibration  process  and 
standards  employed  through  detailed  procedures  and  documentation,  trend  and  control  charts, 
and  full  uncertainty  analysis. 

The  calibration  laboratory  should  have  a  program  in  place  that  provides  periodic 
calibration  of  the  reference  standards  through  a  recall  system.  A  laboratory  in  compliance  with 
this  Standard  would  be  considered  competent. 

2.6. 5.2  Example(s). 

1)  If  the  reference  standard  is  calibrated  by  a  national  or  international  standards 
laboratory,  then  it  is  assumed  that  traceability  is  fulfilled  and  that  adequate 
uncertainty  is  provided.  It  is  up  to  the  user  of  these  standards  to  maintain  the 
integrity  of  such  standards  through  adequate  handling,  trend  and  control  charts,  or 
other  intermediate  checks  as  necessary. 

2)  If  the  reference  standard  is  calibrated  with  an  intrinsic  standard  by  the  same 
laboratory  then  full  adequacy  of  calibration  has  to  be  maintained  in  terms  of 
calibration  processes,  procedures,  documentation,  trend  and  control  charts,  and 
uncertainty  analysis. 


7-28 


RCC  Special  Report:  Development  Of  Electro-Optical  (EO)  Standard  Processes  for  Application,  November  2011 


3)  If  the  reference  standard  is  calibrated  by  a  laboratory  not  identified  as  a  national 
or  international  standards  laboratory,  then  that  laboratory  has  to  be  prepared  to 
show  adequacy  of  the  calibration  process  through  detailed  documentation  of  the 
procedures,  trend  and  control  charts,  and  uncertainty  analysis.  This  laboratory 
also  has  to  be  able  to  prove  full  traceability  for  that  standard. 

2.6.6  Additional  Validation.  Where  relevant,  reference  standards,  measuring  and  test 
equipment  shall  be  subject  to  in-service  checks  between  calibrations  and  verifications. 

2.6.6. 1  Interpretive  Guidance.  In  some  applications,  certain  standards  and  equipment  might 
require  additional  checks  and  verifications  in  addition  to  the  normal  calibrations  performed  based 
on  the  official  recall  system.  Such  checks  help  in  providing  trend  and  control  charts,  improve  the 
maintenance  of  drift  rates,  and  increase  confidence  in  measurement  capability. 

Irrespective  of  the  recall  system,  adequate  trend  and  control  charts  should  be 
maintained  of  the  traceable  reference  standards.  In  some  cases,  the  recall  interval  is  short 
enough.  In  other  cases,  the  laboratory  should  implement  additional  measures  to  ensure  the 
integrity  of  the  standards.  This  applies  especially  to  those  standards  that  exhibit  large  (compared 
to  the  quoted  accuracies)  drift  rates.  In  most  cases,  though,  a  well  monitored  recall  system 
should  suffice  to  guarantee  the  integrity  of  the  standards  and  other  test  and  measuring  equipment 
maintained  by  the  laboratory.  Items  not  needed  for  long  periods  or  only  required  if  and  when  a 
special  test  arises,  could  be  put  into  a  temporary  storage  category  with  the  understanding  that 
they  will  be  calibrated  prior  to  the  next  use. 

2.6. 6.2  Example(s).  Sometimes  a  standards  laboratory  may  elect  to  use  a  check  standard  to 
monitor  a  reference  standard  between  regular  calibrations. 

2.6.7  Reference  Material.  Reference  materials  shall,  where  possible,  be  traceable  to 
national  or  international  standards  of  measurements,  or  to  national  or  international  standard 
reference  materials. 

2. 6. 7.1  Interpretive  Guidance.  Reference  materials  used  in  the  calibration  laboratory  should 
be  traceable  to  some  ultimate  source.  Such  sources  could  be  national  or  international  standards 
laboratories,  or  vendors  whose  products  are  recognized  by  such  laboratories. 

2.6.7. 2  Example(s).  Whenever  reference  materials  are  utilized  in  a  calibration  laboratory, 
their  origin  should  be  clearly  stated.  In  most  cases,  reference  materials  such  as  Standard 
Reference  Materials  (SRMs)  are  available  from  National  Standards  Laboratories  directly  and 
hence  national  traceability  is  assured.  In  other  cases,  materials  might  be  available  from  vendors 
with  clear  explanations  of  how  the  materials  were  proven  to  be  of  the  stated  quality.  In  these 
cases,  some  form  of  traceability  has  to  be  available  with  the  materials. 

2.7  Calibration  Methods  and  Procedures 

2.7.1  Documentation.  The  laboratory  shall  have  documented  instructions  on  the  use  and 
operation  of  all  relevant  equipment,  on  the  handling  and  preparation  of  items,  and  for 
calibrations/verifications,  where  the  absence  of  such  instructions  could  jeopardize  the 
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calibrations/verifications.  All  instructions,  standards,  manuals,  and  reference  data  relevant  to  the 
work  of  the  laboratory  shall  be  maintained  up-to-date  and  be  readily  available  to  the  staff. 

2.7. 1.1  Interpretive  Guidance.  Documentation  of  the  calibration/verification  process  is 
usually  necessary  to  maintain  repeatability  of  measurement(s).  This  includes  documentation  of 
the  process  and  the  necessary  instructions  to  properly  operate  the  standards,  the  device  being 
tested,  and  any  ancillary  equipment  that  directly  affects  the  calibration/verification.  Some  items 
being  calibrated  require  special  handling  and  preparation. 

The  documents  should  be  current  to  support  the  calibration/verification  process. 
Technicians  and  engineers  performing  calibrations  should  be  able  to  access  the  needed 
documentation  within  the  laboratory,  offices  in  close  proximity  to  the  laboratory  or  other  suitable 
location  near  the  workplace.  The  person  responsible  for  performing  the  calibration/verification 
should  not  only  be  able  to  obtain  the  needed  documents,  but  needs  to  know  when  and  what 
changes  have  been  made  that  may  affect  the  process. 

Documented  instructions  do  not  necessarily  mean  hard  copy  documents. 
Documentation  can  be  in  the  form  of  software,  CD  ROM,  microfilm,  or  other  suitable  media. 

2.7. 1.2  Example(s).  The  detail  of  the  instructions  necessary  to  accomplish  a  measurement 
varies  according  to  the  skill  and  experience  of  the  metrology  engineer  or  technician,  the 
complexity  of  the  calibration  being  performed  and  how  often  that  the  calibration  is  performed. 
Entry  level  personnel  use  detailed  instructions  and  well  written  procedures  to  master  each  new 
measurement  they  perform.  A  very  senior  engineer  performing  the  same  procedure  on  a  daily  or 
weekly  basis  may  not  need  to  refer  to  the  documentation  as  often.  However,  should  a  manual  or 
procedure  change  be  made  and  not  properly  annotated,  a  problem  could  arise  that  could  make  the 
calibration/verification  invalid.  Documentation  can  include: 

1)  Calibration  procedures  or  procedure  checklists. 

2)  Equipment  installation,  operator  and  service  manuals. 

3)  Reference  books,  such  as  CRC  Engineering  Tables,  ISO  International 
Vocabulary,  technical  textbooks,  and  equipment  manufacturer's  application  notes. 

2.7.2  Use  of  Calibration  Methods  and  Procedures.  The  laboratory  shall  use  appropriate 
methods  and  procedures  for  all  calibrations/verifications  and  related  activities  within  its 
responsibility  (including,  but  not  limited  to,  sampling,  handling,  transport  and  storage, 
preparation  of  items,  estimation  of  uncertainty  of  measurement  and  analysis  of  calibration  data). 

a.  Calibration  procedures  shall  contain  the  required  range  and  tolerance  or 
uncertainty  of  each  item  or  unit  parameter  being  calibrated  or  verified.  In 
addition,  the  procedures  shall  contain  the  generic  description  of  the  measurement 
standards  and  equipment  needed  with  the  required  parameter,  range,  tolerances  or 
uncertainties,  and  specifications  for  performing  the  measurement  of  the 
calibration  or  verification,  and/or  representative  types  (manufacturer,  model, 
option)  that  are  capable  of  meeting  the  generic  description  for  the  measurement 
standards.  The  procedures  shall  be  consistent  with  the  accuracy  required,  and 
with  any  standard  specifications  relevant  to  the  calibrations/verifications 
concerned. 
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b.  The  laboratory  shall  ensure  that  the  calibration  uncertainties  are  sufficiently  small 
so  that  the  adequacy  of  the  measurement  is  not  affected.  Well  defined  and 
documented  measurement  assurance  techniques  or  uncertainty  analyses  may  be 
used  to  verify  the  adequacy  of  a  measurement  process.  If  such  techniques  or 
analyses  are  not  used,  then  the  collective  uncertainty  of  the  measurement 
standards  shall  not  exceed  25%  of  the  acceptable  tolerance  (e.g.,  manufacturer's 
specification)  for  each  characteristic  of  the  measuring  and  test  equipment  being 
calibrated  or  verified. 

2. 7. 2.1  Interpretive  Guidance.  The  method  of  calibration  employed  should  be  appropriate 
for  the  parameter  being  calibrated/verified.  A  detailed  calibration  procedure  should  be  available 
and  followed  to  the  degree  necessary  for  the  calibration/verification  being  performed.  The  use  of 
well  documented  calibration  procedures  is  desired  in  order  to  maintain  consistency  of  the 
measurement  process  when  performed  at  different  times  and  by  different  operators. 

The  standards  selected  and  their  associated  uncertainties  or  tolerances  should  be  adequate  for  the 
calibration/verification  being  performed. 

2.7.2.2  Example(s). 

1)  An  appropriate  method  or  measurement  technique  is  one  that  meets  the  needs  of 
the  calibration/verification  being  performed. 

2)  Some  possible  sources  of  calibration  procedures  are  manufacturer's  procedures, 
and  military  calibration  procedures.  Example  sources  include  Army  Technical 
Bulletins,  Air  Force  Technical  Orders,  Navy  NAVAIR  publications,  in-house 
developed  procedures,  and  the  Government  Industry  Data  Exchange  Program 
(GIDEP).  All  procedures  need  to  be  evaluated  to  ensure  that  they  meet  the  needs 
of  the  calibration/verification  being  performed.  When  using  the  standards 
prescribed  by  the  procedure  it  is  recommended  that,  the  uncertainties  and  related 
Test  Accuracy  Ratio  (TAR)  be  reviewed. 

3)  It  is  generally  considered  good  practice  to  identify  all  the  sources  of  uncertainty 
which  could  affect  a  measurement  process  and  to  quantify  these  effects.  Many 
laboratories  accomplish  this  by  creating  a  list  of  uncertainties  called  an 
“uncertainty  budget.”  The  magnitude  of  each  uncertainty  is  then  determined  and 
recorded  in  the  budget.  Some  sources  of  uncertainty  listed  in  the  budget  may  be 
found  to  be  insignificant.  This  fact  is  noted  in  the  budget.  Having  identified  and 
quantified  all  relevant  sources  of  uncertainty,  their  degree  of  interdependence 
should  be  evaluated.  There  are  standard  formulae  such  as  the  Welch- 
Satterthwaite  equation  for  expressing  this  relationship.  The  next  step  is  to 
combine  all  the  components  to  produce  a  “combined  standard  uncertainty”.  This 
combination  is  accomplished  by  using  a  precisely  defined  mathematical 
relationship  which  is  selected  according  to  the  number  and  relationship  of  the 
components.  Finally,  depending  upon  the  level  of  confidence  at  which  the 
laboratory  has  chosen  to  operate,  the  “combined  standard  uncertainty”  is 
multiplied  by  a  “coverage  factor”,  k,  which  results  in  the  “Expanded 
Uncertainty”.  If  the  Expanded  Uncertainty  is  reported,  then  the  associated 
coverage  factor,  k,  should  be  included.  A  detailed  explanation  of  the  complete 
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process  of  expressing  measurement  uncertainty  can  be  found  in  the  “ISO  Guide  to 
the  Expression  of  Uncertainty  in  Measurement”  (Reference  2.2i).  As  a  default 
alternative  to  doing  an  uncertainty  analysis,  a  laboratory  may  rely  on  a  Test 
Accuracy  Ratio  (TAR)  of  4: 1.  A  TAR  of  4: 1  means  that  the  tolerance  of  the 
parameter  (specification)  being  tested  is  equal  to  or  greater  than  four  times  the 
combination  of  the  uncertainties  of  all  the  measurement  standards  employed  in 
the  test.  If  it  is  determined  that  the  TAR  is  less  than  4:1,  then  one  of  the  following 
methods  may  be  used:  uncertainty  analysis  as  described  above,  guard-banding, 
widening  the  specification,  or  another  appropriate  method.  (Note:  Some  refer  to 
TARs  as  Test  Uncertainty  Ratios  or  TURs.) 

4)  The  NCSL  Recommended  Practice  RP-3.  Calibration  Procedures  Content  and 
Format,  provides  good  guidance  on  how  a  calibration  procedure  may  be  written. 


2.7.3  Standard  Calibration  Methods  and  Procedures.  Where  methods  are  not  specified,  the 
laboratory  shall,  wherever  practical,  select  methods  that  have  been  published  in  international  or 
national  standards,  those  published  by  reputable  technical  organizations  or  in  relevant  scientific 
texts  or  journals. 

2. 7. 3.1  Interpretive  Guidance.  The  laboratory  performing  the  calibration/verification  should 
be  using  an  acceptable  calibration  method  or  procedure.  If  the  equipment  manufacturer  or  the 
customer  does  not  prescribe  a  method  or  procedure,  then  one  must  be  developed  or  obtained 
from  another  source. 

2.7. 3.2  Example(s).  References  to  measurement  techniques  can  be  found  in  textbooks, 
scientific  papers,  professional  journals,  and  from  the  results  of  measurement  research.  Examples 
include: 

1)  Textbooks. 

2)  International  Standards. 

3)  National  standards:  MIL-STD-461D  and  462D,  American  Society  for  Testing 
and  Materials  (ASTM),  American  National  Standards  Institute  (ANSI),  American 
Society  of  Mechanical  Engineers  (ASME),  Instrument  Society  of  America  (ISA). 

4)  IEEE  proceedings. 

5)  Workshop  and  symposium  proceedings. 

6)  Metrologia. 

7)  NIST/NBS  technical  notes. 

8)  NIST/NBS  special  publications. 

2.7.4  Alternatives.  Where  it  is  necessary  to  employ  methods  that  have  not  been  well 
established,  these  shall  be  subject  to  agreement  with  the  customer,  be  fully  documented  and 
validated,  and  be  available  to  the  customer  or  other  recipients  of  the  relevant  reports. 

2.7.4. 1  Example(s).  Some  techniques  for  validating  measurement  methods  are: 

1)  Technical  reviews  by  others  in  same  industry  or  discipline;  these  often  have  to 
rely  upon  “consensus  methods.” 
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2)  Reviews  by  engineers  with  specialty  experience  in  the  discipline. 

3)  Evaluation  and/or  review  by  scientific  investigation. 

4)  Obtaining  acceptance  by  the  measurement  community,  or  “benchmarking.” 

5)  Customer  verification  that  the  calibration  data  or  acceptance  criteria  provided 
satisfies  the  needs  of  the  application. 

6)  Repeating  the  measurement  by  a  second  method  and  comparing  the  results. 

2.7.5  Sampling  Methods.  Where  sampling  is  carried  out  as  part  of  the  calibration  method, 
the  laboratory  shall  use  documented  procedures  and  appropriate  statistical  techniques  to  select 
the  samples. 

2. 7. 5.1  Interpretative  Guidance.  Where  the  calibration  method  requires  the  use  of  a 
population  or  a  sample  of  a  population  of  measurements,  appropriate  statistical  techniques 
should  be  used. 

Statistical  and  quality  textbooks,  software  programs  and  government  standards  (MIL-STD  105D) 
provide  tables  of  random  numbers  and  procedures  for  selecting  random  samples.  These  same 
references  also  help  determine  how  many  samples  are  necessary  for  a  valid  statistical  test.  The 
selected  method  should  be  documented  and  become  part  of  the  calibration  procedure  and  the 
complete  documentation  package. 

2.7.6  Calculations  and  Data  Transfers.  Calculations  and  data  transfers  shall  be  subject  to 
appropriate  checks. 

2.7.6. 1  Example(s).  Some  appropriate  checks  may  include: 

1)  Review  of  calculations. 

2)  Comparison  to  previous  results. 

3)  Standard  data  sets. 

4)  Review  of  abnormal  and/or  unexpected  results. 

5)  Comparison  of  original  data  sheet  against  the  final  report. 

6)  Investigation  of  out-of-tolerance  results. 

2.7.7  Data  Automation.  Where  computers  or  automated  equipment  are  used  for  the 
capture,  processing,  manipulation,  recording,  reporting,  storage  or  retrieval  of  calibration  data, 
the  laboratory  shall  ensure  that: 

a.  The  requirements  of  this  Standard  are  followed. 

b.  Computer  software  is  documented  and  adequate  for  use. 

c.  Procedures  are  established  and  implemented  for  protecting  the  integrity  of  data. 
Such  procedures  shall  include,  but  are  not  limited  to,  integrity  of  data  entry  or 
capture,  data  storage,  data  transmission  and  data  processing. 

d.  Computer  and  automated  equipment  is  maintained  to  ensure  proper  functioning 
and  provided  with  the  environmental  and  operating  conditions  necessary  to 
maintain  the  integrity  of  calibration  data. 


7-33 


RCC  Special  Report:  Development  Of  Electro-Optical  (EO)  Standard  Processes  for  Application,  November  2011 


e.  It  establishes  and  implements  appropriate  procedures  for  the  maintenance  of 
security  of  data  including  the  prevention  of  unauthorized  access  to,  and  the 
unauthorized  amendment  of,  computer  records. 

2. 7. 7.1  Interpretive  Guidance.  This  section  relates  strictly  to  calibration  data.  When 
automated  procedures  are  used,  the  laboratory  should  ensure  that  the  measurement  is  made 
correctly  and  the  calibration  is  not  invalidated.  The  user  of  the  calibration  results  expects  that 
instruments  calibrated,  data  taken,  and  calculations  accomplished  are  correct,  private  (secure) 
and  have  not  been  tampered  with  or  corrupted. 

1)  The  use  of  automated  calibration  processes  does  not  relieve  the  laboratory  of  its 
responsibility  to  meet  all  of  the  requirements  of  the  Standard. 

2)  This  applies  to  both  purchased  and  internally  developed  software.  Since  the 
Standard  does  not  identify  specific  requirements,  each  laboratory  should  define  its 
own  needs  for  software  documentation  and  adequacy. 

3)  Integrity  means  that  the  data  has  not  been  corrupted  nor  tampered  with. 

4)  Computers  and  peripheral  equipment  used  for  calibration  processes  have 
recommended  environmental  operating  needs  and  routine  maintenance  procedures 
that  ensure  their  continued  operation. 

5)  This  section  of  the  Standard  addresses  the  issue  of  security  against  corruption  of 
data  as  well  as  maintaining  its  confidentiality.  Each  laboratory  should  determine 
the  level  of  security  needed  to  adequately  protect  its  data.  Laboratories  with 
many  different  operators  and  a  more  complex  system  may  require  more  elaborate 
security  measures  than  a  laboratory  with  a  single  operator  and  limited  access. 

2. 7. 7. 2  Example(s). 

1)  None  offered 

2)  Documentation  elements  to  consider: 

a.  Inventory  of  utilized  software 

b.  Records  of  software  performance  tests 

c.  Hardware  operating  system 

d.  Operating  instructions 

e.  Software  verification  procedures  using  known  data  sets 

f.  Program  listing 

g.  Program  flowcharts 

h.  Configuration  control  system 

i.  System  management 

j.  Backup  procedures 

k.  Data  archival 

l.  Password  protection 

m.  Read/Write  protection 

n.  Hierarchical  user  access  scheme 

Documentation  of  software  is  a  requirement  of  the  Standard.  This  documentation 
may  be  within  the  program  or  in  accompanying  documents.  Written  instructions  on  how  to  load, 
execute  and  process  a  particular  program  are  recommended.  Screen  prompts  to  assist  the 
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operator  with  the  steps  of  the  procedures,  connections,  and  any  precautions  are  desirable.  Some 
calibrations  can  be  accomplished  with  a  single  connection  and  the  equipment  can  then  be  left 
unattended  until  the  program  is  complete  and  the  report  printed.  Therefore,  the  level  of  operator 
intervention  is  an  indicator  of  how  extensive  the  operator  instructions  in  the  documentation 
should  be. 


Testing  of  software  provides  objective  evidence  of  its  adequacy.  All  software  should 
be  tested  prior  to  use  and  periodically  thereafter.  One  method  of  testing  or  verifying  the  software 
would  be  to  compare  measurement  values  taken  by  the  automated  process  with  those  taken 
manually.  The  periodic  use  of  a  check  standard  could  also  provide  a  reliability  check  of  the 
software. 


3)  Care  in  the  processing  of  calibration  data  can  be  demonstrated  by  documenting 
periodic  tests  of  the  software  and  computer  system.  Tests  can  be  developed  for 
algorithms  to  ensure  that  calculations  are  being  accomplished  correctly.  Frequent 
backups  of  the  system  discs  can  ensure  that  data  is  recoverable.  Clearly 
identifying  the  latest  revision  of  software  being  used  can  ensure  that  the  latest 
changes  or  corrections  have  been  accomplished.  Maintaining  calibration 
programs  on  a  local  area  network  (LAN)  or  shared  resource  manager  (SRM)  with 
a  designated  system  administrator  can  be  effective  in  reducing  errors  in  handling 
software  and  saving  files. 

Calibration  reports  can  sometimes  have  faults  which  could  indicate  possible 
problems  with  the  calibration  itself  or  with  data  reduction.  Reading  the 
calibration  report  and  reviewing  the  data  can  help  eliminate  problems  which  may 
have  occurred  during  the  calibration. 

4)  None  offered. 

5)  Security  to  protect  the  computer  system,  the  software  and  the  calibration  data  can 
be  demonstrated.  Access  to  program  code  should  be  limited  to  reduce  the 
possibility  of  changes  to  test  points,  limits,  and  data.  System  administrators  and 
programmers  are  usually  authorized  to  change  program  code.  Prohibition  of 
unauthorized  operators  can  be  accomplished  with  a  system  of  log-on  protocols 
and  passwords  that  are  changed  frequently.  A  positive  system  backup  process, 
repeated  frequently,  is  a  good  plan  to  ensure  data  is  not  lost  and  is  recoverable. 

Routine  screening  for  computer  viruses  is  a  good  plan  to  protect  programs  and 
archived  data. 

2.7.8  Consumable  Materials.  Documented  procedures  shall  exist  for  the  purchase, 
reception  and  storage  of  consumable  materials  used  for  the  technical  operations  of  the  laboratory 
that  can  affect  the  results  of  calibrations. 

2.7.8. 1  Interpretive  Guidance.  This  requirement  addresses  procedures  and  not  the  quality  of 
consumables. 

2.7. 8. 2  Excimple(s).  One  method  of  documenting  the  purchase  of  consumables  might  be  to 
use  an  approved  vendor  list  along  with  a  procedure  for  its  use.  Receiving  inspection  of  these 
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products  is  appropriate  to  ensure  that  the  correct  product  has  been  delivered  as  specified  by  the 
purchase  order.  Handling  and  storage  of  this  material  may  require  special  procedures,  especially 
for  flammable,  toxic  or  dangerous  materials.  Where  appropriate,  Material  Safety  Data  Sheets 
(MSDS)  should  be  obtained  and  reviewed.  Examples  of  consumables  are: 

1)  Fluid  for  oil  or  temperature  baths. 

2)  Special  preservatives  for  machined  surfaces. 

3)  Reagent  grade  chemicals. 

4)  Certain  cleaning  compounds  or  fluids. 

5)  Compressed  gases. 

6)  Certain  grades  of  cloth  or  paper  wipes. 

7)  Batteries. 

Material  used  in  the  routine  operation  of  the  laboratory  such  as  office,  hygiene,  and 
housekeeping  products  normally  do  not  require  documented  controls  as  outlined  in  this 
paragraph. 
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2.8  Handling  of  Calibration  Items 

2.8.1  Documentation.  The  laboratory  shall  have  a  documented  system  for  uniquely  identifying 
the  items  to  be  calibrated,  to  ensure  that  there  can  be  no  confusion  regarding  the  identity  of  such 
items  at  any  time. 

2. 8. 1.1  Interpretive  Guidance.  The  laboratory  should  have  a  documented  process  for 
tracking  all  equipment  and  standards  received  for  calibration  for  the  purpose  of  identifying 
ownership,  special  calibration  requirements,  special  handling,  or  other  details  unique  to  each 
item.  This  should  apply  whether  or  not  the  equipment  comes  from  sources  within  the  company 
or  from  sources  outside  the  control  of  the  laboratory  performing  the  calibration. 

2. 8. 1.2  Example(s).  One  common  method  of  uniquely  identifying  items  received  for 
calibration  is  to  generate  a  “work  order”  or  “service  requisition”  document  that  can  be  attached 
to  the  item  and  serve  as  a  job  traveler.  This  document  generally  identifies  the  item  with  such 
information  as  asset  number,  manufacturer,  model,  serial  number,  etc.  It  should  also  provide 
information  that  identifies  ownership  of  the  item.  Other  information  included  on  this  form  might 
be  calibration  procedure  number,  special  calibration  requirements  (limited  calibration: 
calibrating  one  range  of  a  multiple  range  device),  and  billing  information  (purchase  order 
number  etc.). 

2.8.2  Records.  Upon  receipt  of  the  calibration  item,  any  abnormalities  or  departures  from 
standard  condition  as  prescribed  in  the  relevant  calibration  method  shall  be  recorded.  Where 
there  is  any  doubt  as  to  the  item's  suitability  for  calibration,  where  the  item  does  not  conform  to 
the  description  provided,  or  where  the  calibration  required  is  not  fully  specified,  the  laboratory 
shall  consult  the  customer  for  further  instruction  before  proceeding.  The  laboratory  shall 
establish  whether  the  item  has  received  all  necessary  preparation,  or  whether  the  customer 
requires  preparation  to  be  undertaken  or  arranged  by  the  laboratory. 

2.8.2. 1  Interpretive  Guidance.  The  laboratory  should  have  a  documented  system  that 
describes  the  process  of  treating  departures  and  abnormalities  from  the  norm  at  any  point  in  the 
calibration  process  from  receipt  to  return  of  items  to  be  calibrated.  There  should  also  be  a 
procedure  for  documenting  and  bringing  such  abnormalities  to  the  attention  of  the  customer  or 
user  so  that  problems  are  properly  identified  and  if  possible  resolved. 

2. 8.2.2  Excimple(s).  Some  examples  of  abnormal  events  are: 

1)  Physical  damage  during  shipping  or  handling  while  in  transit  from  the  customer  to 
the  laboratory  or  while  in  the  laboratory. 

2)  Items  submitted  for  calibration  that  are  not  capable  of  meeting  requested 
calibration  requirements.  A  customer  or  user  may  ask  more  of  the  calibration 
than  the  item  is  capable  of  meeting.  For  general  purpose  equipment,  some 
laboratories  use  a  “Limited  Calibration”  label  or  tag  to  identify  conditions  that  are 
outside  normal  requirements.  Limitations  on  the  calibration  can  also  be 
documented  in  a  Report  of  Calibration. 
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3)  Equipment  failure  during  the  calibration.  There  should  be  a  definite  procedure 
followed  when  such  events  occur.  All  such  events  should  be  documented  for 
customer  feedback  as  well  as  the  historical  record. 

2.8.3  Handling.  The  laboratory  shall  have  documented  procedures  and  appropriate 
facilities  to  avoid  deterioration  or  damage  to  the  calibration  item,  during  storage,  handling, 
preparation,  and  calibration;  any  relevant  instructions  provided  with  the  item  shall  be  followed. 
Where  items  have  to  be  stored  or  conditioned  under  specific  environmental  conditions,  these 
conditions  shall  be  maintained,  monitored  and  recorded  where  necessary.  Where  a  calibration 
item  or  portion  of  an  item  is  to  be  held  secure  (for  example,  for  reasons  of  record,  safety  or 
value,  or  to  enable  check  calibrations  to  be  performed  later),  the  laboratory  shall  have  storage 
and  security  arrangements  that  protect  the  condition  and  integrity  of  the  secured  items  or  portions 
concerned. 

2.8.3. 1  Interpretive  Guidance.  There  should  be  a  documented  system  in  place  that  describes 
the  procedures  for  protecting  calibration  items  from  loss,  damage,  or  deterioration,  or  other 
condition  that  may  affect  the  integrity  of  the  equipment  or  the  calibration  results.  Customer  or 
user  supplied  handling  and  preparation  instructions  should  be  carefully  followed.  This  applies  to 
both  pre  and  post  calibration.  If  there  are  requirements  to  store  or  handle  items  in  a  specific 
manner,  then  these  requirements  should  be  clearly  recorded  and  transmitted  to  all  those  that  have 
a  need  to  know. 

2.8. 3.2  Example(s).  Long  term  storage  of  equipment  may  require  special  consideration  such 
as  items  with  batteries,  which  should  be  removed  if  there  is  a  possibility  of  corrosion.  Items  put 
into  long  term  storage  which  contain  passive  but  delicate  electronic  components  should  be 
identified  as  requiring  calibration  before  usage. 

Restraint  of  portions  of  an  item  for  safety  reasons  should  not  in  any  way  damage  the  item  or 
affect  the  calibration.  An  item  that  measures  high  pressure  may  have  hoses  that  should  be 
restrained  so  that  if  one  were  to  burst  or  come  loose  there  would  be  no  chance  of  personal  injury. 
High  pressure  devices  should  also  be  behind  protective  shields.  All  devices  that  have  the  ability 
to  cause  injury  should  have  handling  precautions  carefully  considered.  Read  instruction  manuals 
and  consult  with  safety  engineers  regarding  a  specific  application. 

2.8.4  Safety.  The  laboratory  shall  have  documented  procedures  for  the  receipt,  retention  or 
safe  disposal  of  calibration  items,  including  all  provisions  necessary  to  protect  the  integrity  of  the 
laboratory. 

2.8.4. 1  Interpretive  Guidance.  Items  received  for  calibration  should  do  nothing  to  adversely 
influence  any  measurements  in  the  laboratory  if  proper  procedures  are  followed.  Hazardous 
materials  consumed  in  the  calibration  process  should  be  disposed  of  in  accordance  with  local 
safety  or  environmental  regulations. 

2. 8.4.2  Example(s).  Hazardous  materials  should  have  clear  procedures  written  and  followed, 
not  only  to  establish  proper  handling  technique,  but  also  to  specify  what  should  be  done  in  the 
event  of  accidents.  For  example,  items  containing  mercury  (barometers)  should  have  special 
handling  instructions  so  that  there  is  no  possibility  of  a  mercury  spill.  Proper  safeguards  should 
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be  in  place  to  prevent  accidents.  In  addition,  proper  training  of  personnel  with  access  to  the 
laboratory  should  be  provided.  This  includes  people  not  normally  in  the  laboratory  such  as 
clerks  and  after-hours  maintenance  personnel. 

2.8.5  Seals.  Tamper-resistant  seals  shall  be  affixed  to  operator  accessible  controls  or 
adjustments  on  measurement  standards  or  measuring  and  test  equipment  which,  if  moved,  will 
invalidate  the  calibration.  The  laboratory's  calibration  system  shall  provide  instructions  for  the 
use  of  such  seals  and  for  the  disposition  of  equipment  with  damaged  or  broken  seals. 

2. 8. 5.1  Interpretive  Guidance.  Operator  accessible  adjustments  should  be  sealed  so  that  if 
the  adjustment  is  moved,  there  is  an  indication  that  the  calibration  may  not  be  valid.  The  use  of 
seals  on  measurement  equipment  and  standards  provides  a  visual  means  to  the  user  and  customer 
that  the  calibration  adjustments  have  not  been  tampered  with  since  the  last  authorized  entry  or 
calibration. 

2.8. 5.2  Example(s).  Some  schemes  for  accomplishing  this  task  are: 

1)  Calibration  adjustments  may  be  sealed  with  torque  paint. 

2)  Tamper-resistant  seals  may  be  used  over  equipment  covers  or  doors  so  that 
individual  adjustments  need  not  be  sealed. 

3)  Lead  wire  seals  may  be  used  to  seal  doors  on  large  consoles  for  the  same  reason 
listed  above. 

4)  Some  facilities  may  use  dip  seal  on  mechanical  gages  that  are  subject  to  wear,  to 
indicate  usage.  (If  the  gage  has  not  been  used  then  re-calibration  may  not  be 
required). 

Whatever  technique  is  used,  there  should  be  a  clear  procedure  to  be  followed  when 
seals  are  found  broken.  There  should  also  be  a  process  in  place  that  evaluates  the  potential 
impact  on  products  tested  with  equipment  found  to  have  disturbed  adjustments. 

2.9  Records 

2.9.1  Laboratory  Records.  The  laboratory  shall  maintain  a  record  system  to  suit  its  particular 
circumstances  and  comply  with  any  applicable  regulations.  The  records  for  each  calibration 
shall  contain  sufficient  information  to  permit  the  repetition  of  the  calibration.  The  records  shall 
include  the  identity  of  personnel  involved  in  preparation  and  calibration. 

2. 9. 1.1  Interpretive  Guidance.  The  Standard  allows  for  a  flexible  approach  to  the 
maintenance  of  laboratory  records  as  long  as  certain  requirements  are  met.  Specifically,  the 
Standard  requires  that  the  recorded  information  is  adequately  detailed  to  allow  a  calibration  to  be 
repeated,  and  that  the  individual(s)  who  did  the  work  are  identified.  The  individuals  may  have 
been  involved  in  preparation,  and/or  calibration.  Preparation  may  include  activities  which  would 
affect  the  outcome  of  the  calibration  such  as  cleaning  of  the  instrument  to  be  calibrated  or  setting 
up  the  test  equipment.  The  record  system  should  provide  confidence  in  the  validity  of  the  data 
and  support  consistent  results  from  calibration  to  calibration.  The  records  should  be  kept  intact 
and  uncorrupted.  Data  integrity  and  reliability  are  of  the  utmost  importance. 
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Records  may  also  have  to  comply  with  certain  regulations  which  are  specific  to  the 
customer,  contract,  industry,  type  of  instrument  or  its  application,  geographic  location,  or 
agency.  Agencies  may  be  local,  state,  federal,  or  international. 

2.9. 1.2  Excunple(s).  The  laboratory’s  record  system  may  be  physically  (hard  copy)  or 
electronically  generated  and  maintained  depending  on  the  needs  of  the  particular  laboratory,  the 
customer,  and  regulatory  agencies.  Considerations  include  data  integrity,  speed  and  ease  of 
generation  and  maintenance,  data  retrieval  requirements,  volume  of  records,  and  record  retention 
requirements. 

The  laboratory  record  must  include  the  identity  of  personnel  involved  in  the 
preparation  and  calibration.  This  may  be  in  the  form  of  the  individual’s  name,  or  a  uniquely 
assigned  identifier.  The  latter  requires  a  cross-reference  between  the  identifier  and  the  identity 
of  the  individual.  The  integrity  of  the  identification  is  important.  This  may  require  a  signature 
of  the  individual,  an  exclusive  rubber  stamp,  a  computer  password,  or  other  unique  way  of 
assuring  the  correct  identity  of  the  individual  responsible. 

2.9.2  Bookkeeping.  All  records  (including  those  pertaining  to  calibration  equipment), 
certificates,  and  reports  shall  be  safely  stored  and  held  secure  and  in  confidence  to  the  customer 
for  the  period  specified  in  the  quality  manual. 

2.9.2. 1  Interpretive  Guidance.  The  quality  manual  must  clearly  state  the  records  retention 
time  period.  The  information  should  be  treated  as  proprietary  to  the  customer.  Whether  records 
are  maintained  on  paper  or  electronically,  they  should  be  safely  stored  and  protected  from 
accidental  or  inappropriate  alterations  or  destruction. 

Records  may  be  charged  for  the  purposes  of  correction  or  some  other  appropriate 
reason,  but  a  traceable  history  of  these  changes  should  be  preserved.  A  common  practice  to 
accomplish  this  is  to  draw  a  single  line  through  the  material  to  be  changed  and  to  date  and  initial 
the  alterations. 

2. 9. 2. 2  Example(s).  A  specific  policy  for  records  retention  is  recommended.  Retention  may 
be  based  on  a  specific  calendar  period,  on  a  number  of  calibration  cycles,  or  some  other  system 
that  is  well  defined.  The  record  retention  period  should  be  understood  and  acceptable  to 
calibration  customers. 

Hard  copy  documents  should  be  kept  in  a  safe  place  that  does  not  adversely  affect  the 

document. 


Computer  data,  the  disk,  tape  backup  or  other  electronic  media  storage  shall  keep  the 
data  safe  and  retrievable.  Electronic  media  history  should  be  read-only  after  the  calibration 
event.  If  it  is  reasonably  possible  for  computerized  historical  data  to  be  altered  after  the  event  of 
calibration,  a  hard  copy  should  be  kept  as  the  “record  copy.” 
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2.10  Certificates  and  Reports 

2.10.1  Documentation  of  the  Results.  When  a  certificate  or  report  is  issued,  the  results  of  the 
calibration,  or  series  of  calibrations  carried  out  by  the  laboratory  shall  be  accurate,  clear, 
unambiguous  and  objective,  in  accordance  with  any  instructions  in  the  calibration  methods.  The 
results  should  normally  be  reported  in  a  calibration  report  or  certificate  and  shall  include  all  the 
information  necessary  for  the  interpretation  of  the  calibration  results  and  all  information  required 
by  the  method  used. 

2.10.1.1  Interpretive  Guidance.  This  requirement  addresses  the  documentation  of  results  of 
calibration  and  not  if/when  a  certificate  or  report  is  issued.  In  some  cases  no  documentation  is 
provided  other  than  a  calibration  label  affixed  to  or  accompanying  the  device.  The  requirement 
addresses  the  reliability,  clarity,  completeness  and  efficiency  of  documented  results  when 
documentation  is  provided  to  the  customer. 

2.10.1.2  Example(s).  The  Standard  does  not  distinguish  between  a  Calibration  Report  and  a 
Calibration  Certificate.  (Calibration  certificate  or  report:  “Document  that  presents  calibration 
results  and  other  information  relevant  to  a  calibration.”)  When  no  distinction  is  made,  otherwise 
identical  documents  could  be  entitled  either  “Report  of  Calibration”  or  “Certificate  of 
Calibration”  and  be  in  compliance  with  the  Standard.  However,  this  blurring  of  the  distinction 
between  calibration  reports  and  certificates  is  not  universally  followed  either  in  the  US  or 
elsewhere.  One  practice,  used  by  many  calibration  organizations  in  the  US,  is  described  in  the 
NCSL’s  RP-1 1,  which  distinguishes  between  reports  and  certificates.  For  example,  a  document 
which  includes  measurement  data  is  sometimes  called  a  “Report  of  Calibration”  whereas  a 
document  which  states  conformance  to  a  tolerance  specification  is  sometimes  called  a 
“Certificate  of  Calibration”,  even  if  measurement  data  is  also  included. 

2.10.2  Format  and  Contents.  Each  certificate  or  report  shall  include  at  least  the  following 
information: 

1)  A  title,  e.g.  "Calibration  Report"  or  "Calibration  Certificate." 

2)  Name  and  address  of  laboratory,  and  location  where  the  calibration  was  carried 
out  if  different  from  the  address  of  the  laboratory. 

3)  Unique  identification  of  the  certificate  or  report  (such  as  serial  number)  and  of 
each  page,  and  the  total  number  of  pages. 

4)  Name  and  address  of  customer,  where  appropriate. 

5)  Description  and  unambiguous  identification  of  the  item  calibrated. 

6)  Characterization  and  condition  of  the  calibration  item. 

7)  Date(s)  of  performance  of  calibration,  where  appropriate. 

8)  Identification  of  the  calibration  procedure  used,  or  unambiguous  description  of 
any  non-standard  method  used. 

9)  Reference  to  sampling  procedure,  where  relevant. 

10)  Any  deviation  from,  additions  to,  or  exclusions  from  the  calibration  method,  and 
any  other  information  relevant  to  a  specific  calibration,  such  as  environmental 
conditions. 

11)  Measurements,  examinations  and  derived  results,  supported  by  tables,  graphs, 
sketches,  and  photographs,  as  appropriate,  and  any  failures  identified. 
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12)  A  statement  of  the  estimated  uncertainty  of  the  calibration  result  (where  relevant). 

13)  A  signature  and  title  or  an  equivalent  identification  of  the  person(s)  accepting 
responsibility  for  the  content  of  the  certificate  or  report  (however  produced),  and 
date  of  issue. 

14)  Where  relevant,  a  statement  to  the  effect  that  the  results  relate  only  to  the  items 
calibrated. 

15)  A  statement  that  the  certificate  or  report  shall  not  be  reproduced  except  in  full, 
without  the  written  approval  of  the  laboratory. 

16)  Special  limitations  of  use. 

17)  Traceability  statement. 

2.10.2.1  Interpretive  Guidance. 

1)  The  laboratory  may  choose  other  titles  for  its  reports  or  certificates.  The 
essential  point  is  that  a  title  be  included. 

2)  None  offered. 

3)  None  offered. 

4)  None  offered. 

5)  None  offered. 

6)  If  the  device  as  received  appears  to  have  been  dropped  or  otherwise  subjected  to 
improper  handling  or  storage  this  fact  should  be  included  on  the  Certificate  or 
Report  of  Calibration.  If  the  as-received  conditions  were  such  that  contact  with 
the  customer  was  made  prior  to  the  start  or  completion  of  the  calibration  any 
special  comments  or  instructions  from  the  customer  should  be  noted  or 
referenced.  Also  include  whether  the  device  was  in-tolerance  or  out  -of- 
tolerance  as  received. 

7)  Generally,  the  calibration  date  reported  is  the  last  day  during  which 
measurements  were  performed.  Where  calibration  is  completed  substantially 
earlier  than  the  certificate,  or  is  performed  over  several  days,  this  information 
might  be  provided. 

8)  Sufficient  information  should  be  provided  to  allow  for  the  replication  of  the 
calibration. 

9)  There  may  be  times  when  the  results  of  the  calibration  or  verification  on 
samples  of  a  batch  or  lot  of  devices  of  the  same  description  may  be  extended  to 
apply  to  other  devices  in  the  same  batch  or  lot  from  which  the  samples  were 
chosen.  When  this  is  the  case  it  should  be  so  noted  along  with  a  description  or 
reference  to  a  documented  procedure  describing  valid  statistical  or  other 
techniques  for  selection  of  samples. 

10)  The  first  part  of  this  requirement  is  similar  to  h)  above  and  could  be 
accomplished  by  referencing  a  uniquely  identified  procedure  with  exceptions. 
The  second  part  requires  the  reporting  of  influence  quantities  such  as 
temperature,  humidity,  gravity,  etc.,  that  may  affect  the  reported  values  beyond 
the  reported  uncertainties. 

1 1)  Measurements,  examinations,  and/or  derived  results  should  be  supplied  as 
appropriate  to  customer  and  laboratory  requirements.  Any  failures  identified 
will  always  be  supplied.  Calibration  results  may  be  reported  in  one  of  many 
forms.  See  the  examples  which  list  some  possibilities.  Tables,  graphs, 
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sketches,  and  photographs  may  be  supplied  when  appropriate  to  support 
measurements,  examinations,  and  derived  results. 

12)  Where  actual  values  are  reported  (as  requested  by  the  customer  which  may  also 
be  the  laboratory  itself)  the  uncertainty  associated  with  the  value(s)  should  be 
reported. 

13)  The  person  responsible  for  the  report  or  the  certificate,  not  necessarily  the 
person  who  performed  the  calibration,  should  be  readily  identifiable.  The  date 
the  report  is  issued  should  also  be  included. 

14)  Where  necessary  to  avoid  the  implication  that  the  results  may  apply  to  other 
identical  or  related  devices,  a  statement  that  such  is  not  the  case  should  be 
offered. 

15)  None  offered. 

16)  When  a  special  or  limited  calibration  is  performed,  the  Calibration  Report  or 
Calibration  Certificate  should  contain  a  notice  that  alerts  the  user  that  a  special 
limitation-of-use  condition  exists  and  identifies  the  nature  of  the  limitation(s). 

17)  Reports  and  Certificates  should  state  the  traceability  to  national,  international  or 
intrinsic  (reference)  standards.  Where  such  traceability  does  not  exist,  it  should 
state  which  alternative  applies  to  the  calibration  or  verification  being 
documented. 

2.10.2.2  Example(s). 

1)  Other  possible  titles  could  be  Data  Sheets,  Report  of  Calibration,  and  Certificate 
of  Conformance. 

2)  None  offered. 

3)  Some  examples  of  unique  identifiers  for  each  certificate  and/or  each  page  might 
be  a  report  or  certificate  number,  job  number,  sales  order  number,  serial  number 
of  the  equipment,  and  date  of  calibration.  Examples  of  page  numbering  might 
be  “page  1  of  x,”  or  “1/x.”,  or  the  total  number  of  pages  on  cover  page  with 
each  page  numbered. 

4)  Situations  where  a  name  and  address  may  not  be  required  include  classified 
programs,  and  warehouse  products  for  sale. 

5)  Some  examples  could  be  item  nomenclature;  manufacturer,  model,  and  serial 
number;  nomenclature  and  asset  number;  nomenclature  and  property  number; 
nomenclature  and  control  number. 

6)  Some  examples  of  conditions  that  might  be  noted  are  damaged  items,  out-of- 
tolerance,  intolerance,  inoperative,  and  accessories  missing. 

7)  An  example  where  a  range  of  dates  may  be  reported  is  DC  voltage  standards 
usually  require  sets  of  measurements  taken  over  specific  periods  of  time  in 
order  to  determine  the  drift  rate. 

8)  Some  examples  of  identifying  standard  procedures  would  be  to  call  out  its 
procedure  number,  reference  a  section  of  a  manufacturer’s  manual,  reference  an 
industry  or  government  standard  (e.g.,  ASTM,  NCSL,  ASME,  etc.).  Some 
examples  for  identifying  non-standard  methods  are  to  reference  a  standard 
procedure  indicating  any  departures  from  it;  to  explain  in  detail  how  the  method 
was  executed;  then  create  an  ad  hoc  method,  uniquely  identify  it,  and  reference 
it. 
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9)  None  offered. 

10)  An  example  might  be  the  temperature  at  which  a  measurement(s)  was  made  in 
the  calibration  process.  Also  temperature  limits,  not  stated  in  the  procedure  or 
specification,  beyond  which  the  calibration  is  not  valid.  The  latter  example  also 
addresses  a  "special  limitation  of  use"). 

1 1)  Some  examples  of  calibration  results  might  be:  for  single  parameter  units,  a 
pass/fail  statement  may  be  adequate;  a  statement  of  compliance  with  a 
specification,  such  as  statement  that  instrument  meets  manufacturer’s 
requirements;  numerical  measurement  data  only;  measurement  data,  applicable 
correction  factors,  and  appropriate  calculated  results;  deviations  from  nominal. 

12)  Some  examples  where  reporting  an  uncertainty  statement  would  usually  be 
relevant  are  where  values  are  assigned  to  reference  standards;  where  numerical 
results  are  reported  and  no  tolerance  specification  is  identified.  An  example 
where  reporting  an  uncertainty  statement  would  usually  not  be  relevant  is  when 
stating  conformance  with  a  specification  with  or  without  data. 

13)  Some  examples  of  equivalent  identification  may  be  an  identifier  which  is 
appropriately  controlled,  or  a  unique  personal  identification  number.  Some 
examples  of  how  the  identifier  could  be  produced  are  a  rubber  stamp,  a  bar 
code,  and  a  computer-generated  identifier. 

14)  For  example,  when  only  selected  instruments  within  a  multiple  instrument  test 
stand  are  being  calibrated,  the  report  should  indicate  this  situation. 

15)  None  offered. 

16)  Examples  of  such  special  or  limited  calibrations  are: 

a.  Not  all  functions  on  a  multi-function  device  are  calibrated. 

b.  All  ranges  on  a  multi-range  device  not  calibrated. 

c.  May  not  be  used  above  27  °C. 

d.  Performance  is  optimized  on  a  specific  range  or  function. 

17)  None  offered. 
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2.10.3  Subcontract.  Where  the  certificate  or  report  contains  results  of  calibrations  performed 
by  subcontractors,  these  results  shall  be  clearly  identified. 

2.10.3.1  Interpretive  Guidance.  If  all  or  part  of  the  calibration  is  performed  by  a  sub¬ 
contractor,  this  fact  must  also  appear  on  the  report  or  certificate  as  well  as  appropriate  other 
related  information,  e.g.  identification,  location,  etc.  of  the  sub-contractor. 

2.10.4  Standardization.  Particular  care  and  attention  shall  be  paid  to  the  arrangement  of  the 
certificate  or  report,  especially  with  regard  to  presentation  of  the  calibration  data  and  ease  of 
assimilation  by  the  reader.  The  format  shall  be  designed  carefully  and  specifically  for  each  type 
of  calibration  performed;  however,  the  headings  shall  be  standardized  as  much  as  possible. 

2.10.4.1  Interpretive  Guidance.  The  concept  behind  this  section  is  to  promote  the  ease  of  use 
and  review  through  a  combination  of  reasonable  flexibility  and  standardization,  and  to  ensure 
that  necessary  information  unique  to  a  particular  calibration  is  not  omitted.  It  is  considered  good 
practice  to  standardize  elements  common  to  all  reports  generated  within  a  laboratory. 

2.10.5  Amendments.  Material  amendments  to  a  calibration  report  or  calibration  certificate 
after  issue  shall  be  made  only  in  the  form  of  a  further  document,  or  data  transfer  including  the 
statement  "supplement  to  calibration  report  [or  calibration  certificate],  serial  number... [or 
otherwise  identified]",  or  equivalent  form  of  wording.  Such  amendments  shall  meet  all  the 
relevant  requirements  of  this  Standard. 

2.10.5.1  Example(s).  Material  amendments  may  include  revision  of  the  actual  data;  correction 
of  erroneous  calculations,  data  transcription  errors,  erroneous  calibration  dates;  insertion  of 
omitted  data. 

2.10.6  Notification.  The  laboratory  shall  notify  customers  promptly,  in  writing,  of  the 
following: 

1)  Any  event  such  as  the  identification  of  defective  calibration  equipment  that  casts 
doubt  on  the  validity  of  results  given  in  any  calibration  report  or  certificate,  or 
amendment  to  a  report  or  certificate.  Such  notification  shall  quantify  the 
magnitude  of  error  created  in  the  calibration  results. 

2)  Any  customer's  measuring  and  test  equipment  found  significantly  out-of-tolerance 
during  the  calibration/verification  process.  Measurement  data  shall  be  reported  so 
that  appropriate  action  can  be  taken. 

2.10.6.1  Interpretive  Guidance.  The  users  of  M&TE  and  measurement  standards  should  be 
notified  when: 

1)  Any  condition  which  casts  doubt  on  the  validity  of  results  of  any  previous 
calibration  report  or  certificate.  The  latter  might  include  such  occurrences  as 
discovery  of  an  out-of-tolerance  (OOT)  calibration  standard,  errors  in  calibration 
procedure,  or  computer  entry  error,  that  adversely  affects  calibration  results. 

2)  Any  instrument  found  to  be  significantly  OOT.  Agreement  should  be  reached  by 
the  customer  and  the  calibration  laboratory  as  to  what  constitutes  “significantly 
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out-of-tolerance”.  In  most  cases,  the  customer  is  in  a  better  position  to  determine 
what  is  “significant”  because  he  is  aware  of  the  equipment  usage  and  application. 
However,  the  customer  may  wish  to  take  advantage  of  the  calibration  laboratory’s 
expertise  and  experience  in  this  matter.  Therefore,  experience  has  shown  that  the 
best  decisions  are  made  by  mutual  agreement  on  this  topic.  If  such  agreement  is 
not  possible,  the  laboratory  should  have  a  documented  policy  on  this  issue.  The 
purpose  of  the  above  notifications  is  to  alert  the  customer  of  any  condition  which 
could  have  caused  erroneous  measurements.  This  enables  the  customer  to  take 
appropriate  corrective  action.  For  example,  M&TE  or  measurement  standards 
would  be  considered  "significantly"  OOT  when  their  condition  could  adversely 
affect  product  quality,  measurement  integrity,  or  safety.  The  specific  parameter 
and  magnitude  of  the  OOT  condition  which  constitute  adverse  consequences  is 
dependent  upon  the  use  and  application  of  the  M&TE  or  measurement  standard. 

2.10.6.2  Excunple(s).  Instances  which  may  cast  doubt  on  previous  calibrations  should  be 
addressed  by  the  laboratory.  Examples  of  such  occurrences  may  include: 

1)  Later  discovery  of  erroneous  data  transcription. 

2)  Calibration  standards  later  found  OOT. 

3)  Computer  software  errors. 

4)  Later  discovery  of  calibration  procedure  errors. 

One  approach  to  defining  “significant  OOT”  for  a  laboratory  may  require  reporting  of 
all  OOT  M&TE  and  measurement  standards  systems  to  the  user  and/or  customer  if  appropriate. 
In  this  case,  determination  of  significance  of  the  OOT  condition  is  then  left  to  the  user  and/or  the 
customer. 


The  laboratory  may  alternatively  ask  the  user  and/or  customer  to  specify  in  advance 
of  calibration  which  instruments,  which  parameters,  and  what  degree  of  OOT  would  result  in 
adverse  effect  on  product  quality,  measurement  integrity,  or  safety.  The  laboratory  may  then 
report  only  those  "significant"  OOT  occurrences. 

In  addition  to  analysis  of  the  use  requirements  of  the  OOT  instrument,  the  ratio  of  the 
OOT  condition  to  the  accuracy  requirement  of  the  instrument  should  be  considered.  For 
example,  if  the  test  accuracy  ratio  between  product  parameter  and  M&TE  is  10: 1  (i.e.  ±10  volts 
for  the  product  parameter  and  ±1  volt  for  the  M&TE),  an  M&TE  OOT  condition  of  2.5  volts 
results  in  an  effective  test  accuracy  ratio  of  4:1  (i.e.  10  volts  for  the  product  parameter  and  2.5 
volts  for  the  M&TE).  This  should  have  little  effect  on  the  testing  results  of  the  product 
parameter.  On  the  other  hand,  if  a  test  accuracy  ratio  of  5:1  between  the  product  parameter  and 
the  M&TE  is  initially  established,  and  the  M&TE  OOT  condition  was  2.5  volts,  the  effective  test 
accuracy  ratio  would  degrade  to  2:1.  This  may  significantly  impact  the  testing  results  of  the 
product  parameter.  The  laboratory  may  consider  this  to  decide  the  "significance"  of  OOT 
conditions  whether  the  decisions  are  made  before  or  after  the  fact.  An  effective  test  accuracy 
ratio  between  M&TE  and  product  parameter  less  than  that  required  by  the  customer  is  not 
acceptable. 
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2.10.7  Transmission.  The  laboratory  shall  ensure  that,  where  customers  require  transmission 
of  calibration  results  by  telephone,  telex,  facsimile  or  other  electronic  or  electromagnetic  means, 
staff  will  follow  documented  procedures  that  ensure  that  the  requirements  of  this  Standard  are 
met  and  that  confidentiality  is  preserved. 

2.10.7.1  Interpretive  Guidance.  Electronic  or  electromagnetic  transmission  of  results  does  not 
alleviate  the  laboratory’s  responsibility  to  issue  a  full  report  which  meets  the  requirements  of  the 
applicable  sections  of  the  Standard.  Electronic  or  electromagnetic  transmission  of  calibration 
results  can  jeopardize  the  confidentiality.  The  laboratory  should  have  documented  procedures 
which  address  this  issue. 

2.10. 7.2  Example(s).  Some  methods  or  techniques  which  minimize  the  risk  to  confidentiality 
are  appropriately  annotated  fax  cover  sheets,  back-up  fax  transmissions  with  a  voice  telephone 
contact  to  verify  receipt,  calling  the  intended  recipient  prior  to  transmissions. 

2.11  Sub-Contracting  of  Calibration 

2.11.1  Sub-Contractor’s  Qualification  and  Customer  Notification.  Where  a  laboratory  sub¬ 
contracts  any  part  of  the  calibration,  this  work  shall  be  placed  with  a  laboratory  complying  with 
the  requirements  of  this  Standard.  The  laboratory  shall  ensure  and  be  able  to  demonstrate  that  its 
sub-contractor  is  competent  to  perform  the  activities  in  question  and  complies  with  the  same 
criteria  of  competence  as  the  laboratory  with  respect  of  the  work  being  sub-contracted.  The 
laboratory  shall  advise  the  customer  of  its  intent  to  subcontract  any  portion  of  the  calibration  to 
another  party. 

2.11.1.1  Interpretive  Guidance.  All  aspects  of  the  calibration  must  conform  to  this  Standard, 
including  work  that  is  sub-contracted.  The  laboratory  is  responsible  to  assure  compliance  by  the 
sub-contracted  laboratory  and  maintain  objective  evidence  of  that  assurance.  Notification  of  the 
intent  to  sub-contract  the  work  should  take  place  prior  to  the  calibration.  If  using  questionnaires 
as  a  means  to  audit  a  sub-contractor’s  quality,  the  examination  of  the  results  from  recent  sub¬ 
contractor  site  visits  and  actual  artifact  tests  should  be  considered  as  a  means  of  strengthening 
the  use  of  this  medium  as  evidence  of  compliance. 

2.11.1.2  Example(s).  There  are  several  elements  which  could  aid  in  collecting  evidence  that  a 
sub-contractor  is  in  compliance  with  the  Standard.  A  few  examples  are: 

1)  On-site  surveys  or  audits  of  the  sub-contractor  by  the  laboratory. 

2)  On  site  surveys  or  audits  of  the  sub-contractor  by  another  qualified  auditing 
organization. 

3)  Use  of  questionnaire  surveys  of  the  sub-contractor. 

4)  Examination  of  the  sub-contractor’s  quality  manual. 

5)  Determination  that  the  sub-contractor  is  accredited  by  an  appropriate  accrediting 
agency. 

6)  Examination  of  the  results  from  the  sub-contractor’s  participation  in  measurement 
assurance  and  laboratory  intercomparison  programs. 
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Accreditation  of  the  sub-contractor  against  the  requirements  of  the  Standard  is 
generally  the  preferred  means  by  which  the  laboratory  can  be  assured  that  the  sub-contractor  is  in 
compliance.  However,  some  of  the  other  approaches  listed  above,  or  combinations  of  them,  may 
prove  to  be  adequate.  For  example,  a  laboratory  may  request  that  a  sub-contractor  complete  a 
questionnaire  survey,  examine  the  subcontractor’s  quality  manual,  and  require  that  the  sub¬ 
contractor  participate  in  a  measurement  assurance  program.  The  laboratory  might  then 
determine  the  sub-contractor’s  compliance  to  the  Standard  through  the  combination  of  all  three 
elements. 

2.1 1.2  Documentation.  The  laboratory  shall  record  and  retain  details  of  its  investigation  of 
the  competence  and  compliance  of  its  sub-contractors  and  maintain  a  register  of  all  sub¬ 
contracting. 

2.11.2.1  Excunple(s).  A  log  or  register  of  sub-contractors  is  commonly  used  for  this 
requirement  of  the  Standard.  The  documentation  from  the  investigation  must  be  traceable  to  the 
log  or  register  and  is  compliance  rated. 

2.12  Outside  of  Support  Services  and  Supplies 

2.12.1  Quality  Assurance.  Where  the  laboratory  procures  outside  services  and  supplies,  other 
than  those  referred  to  in  this  Standard,  in  support  of  calibrations,  the  laboratory  shall  use  only 
those  outside  support  services  and  supplies  that  are  of  adequate  quality  to  sustain  confidence  in 
the  laboratory's  calibrations. 

2.12.1.1  Interpretive  Guidance.  The  laboratory  should  procure  only  those  outside  support 
services  and  supplies  conforming  to  all  requirements  to  assure  compliance  to  all  calibration 
requirements.  Where  there  is  no  specific  quality  requirement  for  the  required  outside  goods  and 
services,  the  laboratory  should  use  acceptable  common  industry  standards. 

2.12.1.2  Example(s).  If  a  piece  of  equipment  is  to  be  free  moving  and  requires  lubrication  but 
no  type  of  lubricant  is  specified,  the  choice  should  be  appropriate  to  the  performance  and 
application  requirements  of  the  instrument  without  compromising  either.  Other  examples  might 
include: 

1)  Oils  for  temperature  baths. 

2)  Mercury  for  barometers. 

3)  Pure  chemicals  and  gases. 

4)  Equipment  servicing,  not  to  include  calibration. 

2.12.2  Quality  Validation.  Where  no  independent  assurance  of  the  quality  of  outside  support 
services  or  supplies  is  available,  the  laboratory  shall  have  procedures  to  ensure  that  purchased 
equipment,  materials  and  services  comply  with  specified  requirements.  The  laboratory  should 
ensure,  wherever  possible,  that  purchased  equipment  and  consumable  materials  are  not  used  until 
they  have  been  inspected,  calibrated  or  otherwise  verified  as  complying  with  any  standard 
specifications  relevant  to  the  calibrations  concerned. 

2.12.2.1  Interpretive  Guidance.  The  laboratory  quality  manual  should  require  that  the 
identification  of  specific  parameters  for  acceptance  be  defined  either  in  the  purchase  order  to  the 
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supplier  of  goods  or  services  and/or  separate  specifications,  prints  etc.  A  clear  understanding  of 
the  characteristics  of  size,  performance  or  other  definable  parameters  and  acceptance  criteria 
when  the  products  or  services  are  received  by  the  laboratory  are  essential  for  adequate 
conformance  verification.  The  use  of  products  that  have  not  been  inspected  or  service 
parameters  that  have  not  been  verified  may  compromise  the  laboratory’s  ability  to  provide 
adequate  data  and  history  of  the  calibration  or  test. 

2.12.3  Record  Keeping.  The  laboratory  shall  maintain  records  of  all  suppliers  from  whom  it 
obtains  support  services  or  supplies  required  for  calibrations. 

2.13  Complaints 

2.13.1  Documentation.  The  laboratory  shall  have  documented  policy  and  procedures  for  the 
resolution  of  complaints  received  from  customers  or  other  parties  about  the  laboratory's 
activities.  A  record  shall  be  maintained  of  all  complaints  and  of  the  actions  taken  by  the 
laboratory. 

2.13.1.1  Interpretive  Guidance.  The  laboratory  should  have  documented  policies  and 
procedures  for  addressing  complaints  against  its  activities.  The  laboratory's  activities  may  be 
considered  as  any  activity  that  relates  to  the  quality  of  the  service  provided,  including,  but  not 
limited  to,  calibrations,  consultations,  advertisement  of  its  services,  or  advertisement  of  its 
accredited  status  (if  applicable). 

2.13.1.2  Example(s).  The  laboratory's  system  may  be  considered  adequate  when  the  policies 
and  procedures  require  some  type  of  record  be  maintained  to  document  all  complaints  received 
and  subsequent  actions  taken  to  resolve  those  complaints.  The  system  used  to  address  and 
resolve  complaints  can  be  as  simple  or  as  elaborate  as  needed.  Simple  log  books  may  suffice  in 
some  operations  and  a  complex  computer  data  base  may  be  needed  in  others.  The  complaint 
system  may  be  tailored  to  the  size  of  the  calibration  operation,  type  of  customers  serviced, 
contract  requirements,  or  other  criteria. 

2.13.2  Actions.  Where  a  complaint,  or  any  other  circumstance,  raises  a  concern  regarding 
the  laboratory's  compliance  with  the  laboratory's  policies  or  procedures,  or  with  the  requirements 
of  this  Standard  or  otherwise  concerning  the  quality  of  the  laboratory's  calibrations,  the 
laboratory  shall  ensure  that  complaints  in  those  areas  of  activity  and  responsibility  involved  are 
promptly  resolved. 

2.13.2.1  Interpretive  Guidance.  The  laboratory's  documented  policy  and  procedures  should 
provide  for  the  prompt  resolution  of  all  complaints  regarding  the  quality  of  the  laboratory's 
activities. 

2.13.2.2  Example(s).  The  laboratory’s  documented  policies  and  procedures  regarding 
corrective  actions  may  range  from  changing  an  internal  procedure  to  recalling  previously 
calibrated  equipment.  Internal  policies  and  procedures  dealing  with  compliance  and  quality 
issues  should  provide  guidance  for  the  resolution  of  all  such  issues  as  soon  as  possible  and 
practical.  Prompt  resolution  not  only  addresses  the  immediate  complaint,  it  also  attempts  to 
identify  and  resolve  the  root  cause  of  the  complaint  (i.e.:  What  went  wrong?  How  do  we  prevent 
it  from  happening  again?). 
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3  PART  III: QUALITY  ASSURANCE  REQUIREMENTS  FOR 
MEASURING  AND  TEST  EQUIPMENT  (M&TE) 

Part  III  applies  to  the  control  of  measuring  and  test  equipment  (M&TE)  used  to  assure 
that  supplies  and  services  comply  with  prescribed  customer  requirements. 

3.1  General  Requirements 

3.1.1  Documentation.  The  supplier  shall  establish  and  document  a  system  to  control  the 
calibration/verification  of  M&TE. 

3.1.2  Validation.  M&TE  used  to  determine  compliance  with  technical  specifications  shall 
be  calibrated  or  verified  in  accordance  with  PART  II  of  this  Standard. 

3.1.3  Recall  and  Verification  Program  The  supplier  shall  have  a  program  to  recall  for 
calibration  or  verification,  or  remove  from  service,  M&TE  that  has  exceeded  its  calibration 
interval,  has  broken  calibration  seals,  or  is  suspected  to  be  malfunctioning  because  of 
mishandling,  misuse,  or  unusual  measurement  results. 

3.1.4  Periodic  Verification.  All  operations  performed  by  the  supplier  in  compliance  with 
this  Standard  shall  be  subject  to  verification  at  unscheduled  intervals. 

3.1.5  Periodic  Auditing.  The  supplier  shall  carry  out  or  arrange  to  be  carried  out,  periodic 
quality  auditing  of  the  calibration  and  verification  system  in  order  to  ensure  its  continuing 
effective  implementation  and  compliance  with  the  requirements  of  this  Standard. 

3. 1.5.1  Assessments  and  Reviews.  Based  on  the  results  of  the  audits,  assessments  and  any 
other  relevant  factors,  such  as  customer  feedback,  the  supplier  shall  review  and  modify  the 
system  as  necessary. 

3. 1.5.2  Documentation.  Plans  and  procedures  for  the  audits  or  assessments  shall  be 
documented.  The  conduct  of  the  audit  or  assessment  and  any  subsequent  corrective  action  shall 
also  be  documented. 

3.2  Detailed  Requirements 

3.2. 1  Calibration  System  Description.  1 15The  supplier  shall  provide  and  maintain  a  written 
description  of  the  calibration/verification  system  covering  M&TE.  The  description  shall  be 
sufficient  to  satisfy  each  requirement  of 

PART  II  of  this  Standard  and  any  deviations  shall  be  submitted  with  supporting  justification  to 
the  customer  for  approval. 

3.2. 1.1  Interpretive  Guidance.  Documentation  of  all  existing  or  proposed  procedures  and 
practices  implementing  the  supplier's  calibration/verification  system  is  necessary  to  promote 
uniformity  of  understanding,  continuity  of  operations  when  personnel  changes  occur  and  to 
provide  a  means  of  demonstrating  the  supplier's  intent,  criteria  and  approach  in  satisfying  the 
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requirements  of  the  Standard.  The  written  description  of  the  supplier's  calibration/verification 
system  should  be  relevant  and  contain  complete  current  details  of  the  system. 

3. 2. 1.2  Excmiple(s).  The  system  description  will  describe  the  requirement,  identify 
responsibility,  and  provide  a  method,  procedure,  or  standard  practice,  including  quantification 
where  appropriate,  for  each  specific  requirement  of  the  Standard.  Any  planned  deviations  by  the 
supplier  to  the  detailed  requirements  of  the  Standard  are  documented.  The  system  description 
provides  instructions  for  obtaining  Customer  review  and  approval  of  deviations  from  the 
Standard  when  required  by  the  contract.  A  standardized  format  for  the  written 
calibration/verification  system  description  is  not  required;  documentation  used  in  the  inspection 
or  quality  program  is  generally  appropriate.  The  supplier  should  ensure  that  changes  to  the 
calibration/verification  system  are  reflected  in  the  system  description  using  some  form  of 
document  control  system. 

3.2.2  Adequacy  of  Measurement  Standards.  Measurement  standards  used  by  the  supplier 
for  calibrating  M&TE  and  other  measurement  standards  shall  comply  with  the  requirements  of  2 
of  the  Standard. 

3.2.2. 1  Interpretive  Guidance.  See  Section  2  of  the  Handbook. 

3.2.22  Example(s).  See  Section  2  of  the  Handbook. 

3.2.3  Environmental  Conditions.  M&TE  shall  be  used  in  an  environment  controlled  to  the 
extent  necessary  to  ensure  valid  results.  Due  consideration  shall  be  given  to  temperature, 
humidity,  lighting,  vibration,  dust  control,  cleanliness,  electromagnetic  interference  and  any 
other  factors  affecting  the  results  of  measurements.  When  pertinent,  these  factors  shall  be 
monitored  and  recorded  and,  when  appropriate,  correcting  compensations  shall  be  applied  to 
measurement  results. 

3.2. 3.1  Interpretive  Guidance.  Environmental  conditions,  such  as  temperature,  relative 
humidity,  vibration,  electrical  interference,  dust,  etc.,  which  could  affect  the  accuracy,  stability, 
or  precision  of  M&TE  require  control  by  the  supplier.  The  degree  of  control  necessary  and  the 
extent  of  compensation,  when  the  required  control  is  not  achieved,  are  directly  related  to  the 
impact  of  the  environmental  conditions  on  the  specific  use  requirements  of  the  M&TE.  As  a 
quality  assurance  measure,  environmental  conditions  should  be  monitored  and  recorded. 

3.2.32  Example(s).  All  M&TE  is  used  in  an  area  environmentally  controlled  to  the  degree 
necessary  to  assure  accurate  measurements.  The  supplier's  written  calibration/verification 
system  description  describes  the  environmental  control  requirements  and  how  compensation, 
when  such  controls  are  not  met,  is  identified  and  accomplished.  In  specific  instances  where 
defined  environmental  control  requirements  are  not  met,  the  supplier  can  demonstrate  that  such 
controls  are  not  required  or  that  he  has  adequately  compensated  for  the  lack  of  control  through 
the  use  of  corrections  or  other  means. 

3.2.4  Intervals  of  Calibration  and  Verification.  M&TE  requiring  calibration  shall  be 
calibrated  or  verified  at  periodic  intervals  established  and  maintained  to  assure  acceptable 
reliability,  where  reliability  is  defined  as  the  probability  that  M&TE  will  remain  in  -  tolerance 
throughout  the  interval.  Intervals  shall  be  established  for  all  M&TE  requiring  calibration  unless 
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the  equipment  is  regularly  monitored  through  the  use  of  check  standards  in  a  documented 
measurement  assurance  process.  Check  standards  must  closely  represent  the  item  parameters 
normally  tested  in  the  process  and  the  check  standard  must  be  verified  periodically.  Where 
intervals  are  used  to  ensure  reliability,  the  interval  setting  system  must  be  systematically  applied 
and  shall  have  stated  reliability  goals  and  a  method  of  verifying  that  the  goals  are  being  attained. 
Intervals  may  be  based  on  usage  or  time  since  last  calibration  or  verification.  All  exemptions 
from  periodic  calibration  or  verification  shall  be  documented.  The  recall  system  may  provide  for 
the  temporary  extension  of  the  calibration  due  date  for  limited  periods  of  time  under  specified 
conditions  that  do  not  unreasonably  impair  the  satisfaction  of  the  customers  requirements. 

3.2.4. 1  Interpretive  Guidance.  The  underlying  basis  for  periodic  calibration  intervals  is  to 
establish  some  usage  or  time  interval  where  there  is  a  high  probability  that  the  instruments  being 
calibrated  will  remain  in-tolerance  throughout  the  established  interval.  If  this  occurs,  then  the 
requirements  of  acceptable  accuracy  and  reliability  are  met.  Since  the  condition  (in-tolerance  or 
out-of-tolerance)  of  M&TE  during  the  calibration  interval  is  unknown,  an  inference  regarding 
their  condition  must  be  made  based  upon  their  condition  found  at  the  end  of  the  interval. 

One  technique  used  in  the  control  of  calibration  intervals  to  achieve  acceptable 
accuracy  and  reliability  is  a  requirement  for  recall  or  verification  at  some  established  interval, 
continuing  review  and  adjustment  of  the  interval  (lengthening  or  shortening)  as  required  to 
maintain  the  acceptable  reliability  (also  called  the  reliability  target),  and  removal  from  use  of 
M&TE  that  do  not  meet  the  reliability  target  after  interval  adjustment. 

A  check  standard  is  some  device  or  instrument  that  is  similar  in  behavior  to  the  items 
being  submitted  for  calibration  by  the  process  that  is  to  be  controlled.  By  making  regular 
measurements  on  one  or  more  check  standards  that  are  known  to  be  stable  and  reproducible,  and 
plotting  the  data  on  a  control  chart,  one  can  observe  graphically  any  changes  in  the  performance 
of  the  controlled  process  and  M&TE.  Statistical  tests  may  then  be  used  to  determine  the 
significance  of  any  change  in  the  behavior  of  the  process  and/or  the  M&TE 

Criteria  for  removal  or  repair  of  instrument  or  compensation  for  environmental 
condition(s)  should  be  established  based  on  the  results  of  the  statistical  test. 

In  the  most  rigorous  approach  to  controlling  the  measurement  process  one  makes  a 
measurement  on  a  check  standard  each  time  a  measurement  is  performed.  The  measured  values 
of  the  check  standard  are  then  plotted  on  a  control  chart  and  analyzed  to  determine  if  the  process 
is  in  a  state  of  statistical  control.  If  the  measurement  on  the  check  standard  indicates  an  out-of¬ 
control  condition,  the  measurement  system  is  flawed  and  must  be  validated  after  the  source  of 
trouble  is  identified  and  corrected. 

3. 2. 4. 2  Example(s).  The  supplier's  written  calibration/verification  system  description 
defines  what  constitutes  an  acceptable  probability  for  the  instruments  being  in-tolerance 
(reliability  target)  during  the  calibration  interval,  how  this  probability  is  determined  and  the 
procedure  for  adjusting  the  interval  when  the  reliability  target  is  not  met. 
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For  some  fixed  value  M&TE,  the  in-tolerance  or  reliability  target  criteria  may  be 
stated  in  terms  of  deviation  from  the  last  calibration  value  rather  than  deviation  from  a  specific 
nominal  value. 

There  are  no  restrictions  as  to  what  methodology  is  employed  to  initially  establish  a 
calibration  interval  to  meet  a  specific  reliability  target  or  how  the  established  interval  is  adjusted 
if  the  desired  reliability  is  not  being  achieved. 

Whatever  methods  are  selected,  the  supplier  can  demonstrate  how  each  M&TE  and 
measurement  standard,  or  homogeneous  group  thereof,  meets  the  reliability  target  through  use  of 
the  established  calibration  interval  and  how  intervals  have  been  adjusted  when  reliability  targets 
were  not  met. 

Specific  reliability  targets  for  M&TE  may  be  established  contractually  or  are 
determined  by  the  supplier  based  on  analysis  of  all  pertinent  factors. 

The  supplier  may  elect  to  use  calibration  intervals  established  by  another  laboratory. 
However,  the  supplier  should  be  aware  of  the  underlying  reliability  targets  for  these  intervals  and 
assure  that  they  are  compatible  with  all  appropriate  requirements. 

All  M&TE  is  included  in  the  supplier's  recall  system. 

Intervals  are  typically  established  in  terms  of  calendar  time  months  or  days,  usage,  or 
a  combination  of  both.  Some  instruments  (usually  measurement  standards  and/or  check 
standards  may  be  calibrated  on  a  continuing  basis  as  part  of  a  measurement  system  under 
statistical  process  control  (SPC)  and  will  not  have  a  finite  calibration  interval. 

If  the  intervals  are  established  on  a  usage  basis,  the  supplier  will  define  what 
constitutes  usage.  In  any  case,  the  supplier  should  maintain  historical  records  for  all  calibrated 
M&TE  to  the  extent  that  the  adjustment  of  intervals  can  be  verified. 

The  supplier's  recall  system  will  include  instructions  that  provide  for  the  following: 

1)  Recall  of  M&TE  for  calibration  according  to  predetermined  schedules  when  other 
control  methods  are  not  used. 

2)  Prompt  release  by  the  user  of  instruments  recalled  for  calibration. 

3)  The  issuing  of  calibration  overdue  notices. 

4)  The  prevention  of  use,  through  impounding  or  other  means,  of  overdue 
instruments. 

The  supplier's  calibration/verification  system  may  include  provisions  for  a  temporary 
extension  of  a  calibration  due  date  for  a  limited  period  of  time  under  certain  specific  conditions, 
such  as  completion  of  a  test  in  progress.  If  such  provisions  are  included,  the  temporary  extension 
must  have  an  adequate  and  documented  basis  and  be  controlled. 

Since  the  supplier  has  the  ability  to  plan  the  use  of  M&TE,  such  extensions  should  be 
the  exception  rather  than  the  rule. 
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The  supplier's  calibration/verification  system  will  also  provide  for  identification  of 
M&TE  that  does  not  meet  (or  cannot  meet)  the  reliability  target  after  interval  adjustment.  Such 
instruments  should  have  their  use  restricted  or  be  removed  from  use. 

3.2.5  Calibration  Procedures.  Procedures  used  to  calibrate/verify  the  suppliers  M&TE  shall 
comply  with  the  requirements  of  2  of  this  Standard. 

3.2. 5.1  Interpretive  Guidance.  See  Section  2  of  this  Handbook. 

3.2. 5.2  Example(s).  See  Section  2  of  this  Handbook. 

3.2.6  Out-of-tolerance  Conditions.  If  any  M&TE  is  found  to  be  significantly  out  of 
tolerance  during  the  calibration/verification  process,  the  suppliers  system  shall  provide  for 
notification  to  the  user  and  appropriate  quality  element  of  the  out-of-tolerance  condition  with  the 
associated  measurement  data  so  that  appropriate  action  can  be  taken. 

3. 2. 6.1  Interpretive  Guidance.  The  supplier's  written  calibration/verification  system 
description  will  include  a  procedure  which  provides  that  the  user  of  M&TE  be  notified  of  any 
instrument  found  to  be  significantly  out-of-tolerance  during  calibration.  The  purpose  of  the 
notification  is  to  alert  the  user  to  the  possibility  of  erroneous  measurements  made  by  the 
significantly  out-of-tolerance  instruments. 

The  supplier's  procedure  providing  for  notification  of  significantly  out-of-tolerance 
instruments  will  also  define  what  constitutes  "Significantly"  out-of-tolerance. 

3. 2. 6.2  Example(s).  The  supplier's  written  calibration/verification  system  description 
includes  the  requirement  for  recording  and  reporting  significantly  out-of-  tolerance  conditions 
identified  during  calibration  of  M&TE.  The  supplier's  calibration/verification  system  includes  a 
reporting  format  which  provides  for  a  description  of  the  specific  out-of-tolerance  condition  with 
supporting  data,  and  the  reporting  channels  to  be  followed.  As  a  minimum,  the  reporting 
channels  should  include  the  instrument  user  and  appropriate  management  responsible  for  the 
quality  of  the  measurements. 

The  supplier's  written  calibration/verification  system  description  describes  what 
constitutes  a  "significant"  out-of-tolerance  condition. 

M&TE  should  be  considered  "significantly"  out-of-tolerance  when  their  condition 
could  adversely  affect  product  quality,  measurement  integrity,  or  safety.  The  specific  parameter 
and  degree  of  out-of-tolerance  condition  which  constitute  adverse  performance  are  largely 
dependent  upon  the  use  and  application  of  the  M&TE.  Accordingly,  determination  of  a 
"significantly"  out-of-tolerance  condition  is  generally  based  on  an  engineering  analysis  of  the 
instrument's  use  requirements,  the  effect  of  the  out-of-tolerance  condition  on  the  parameter  being 
measured  or  calibrated,  and  ultimately  the  effect  on  the  end-product  or  material  being  delivered. 

The  supplier's  calibration/verification  system  description  may  establish  a  reporting 
system  to  notify  the  user  and  quality  elements  of  all  out-of-tolerance  M&TE.  The  determination 
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of  whether  the  out-of-tolerance  condition  significant  or  not  is  then  left  with  the  user  and/or  the 
quality  component  who  have  received  the  notice.  However,  if  it  can  be  determined  beforehand 
which  specific  instruments,  which  specific  parameters,  and  what  degree  of  out-of-tolerance 
would  result  in  an  adverse  effect  on  product  quality,  measurement  integrity,  or  safety,  then  only 
"significantly"  out-of-tolerance  notices  need  be  provided.  In  the  first  instance,  the  supplier 
reports  all  out-of-tolerance  conditions  and  leaves  it  up  to  the  recipient  of  the  notice  to  make  the 
decision  as  to  "significant"  or  not.  In  the  second  instance,  preliminary  efforts  are  expended  to 
determine  what  will  constitute  "significant"  out-of-tolerance  conditions,  and  only  those  so 
established  are  reported  when  they  occur. 

In  addition  to  the  aforementioned  engineering  analysis  for  determining  whether  the 
out-of-tolerance  condition  is  "significant"  or  not,  consideration  should  be  given  to  the  magnitude 
of  the  out-of-tolerance  condition  as  contrasted  with  the  accuracy  requirement  of  the  out-of¬ 
tolerance  instrument.  For  example,  if  the  test  accuracy  ratio  between  product  parameter  and 
M&TE  were  10:1  (e.g.,  ±10  volts  for  the  product  parameter  and  ±1  volt  for  the  M&TE),  an 
M&TE  out-of-tolerance  condition  of  2.5  volts  would  result  in  an  effective  test  accuracy  ratio  of 
4:1  (i.e.  ±10  volts  for  the  product  parameter  and  ±2.5  volts  for  the  M&TE),  which  should  have 
little  effect  on  the  testing  results  of  the  product  parameter.  On  the  other  hand,  if  a  test  accuracy 
ratio  of  5:1  between  product  parameter  and  M&TE  were  initially  established,  and  the  M&TE 
out-of-tolerance  condition  was  again  2.5  volts,  then  the  effective  test  accuracy  ratio  would 
degrade  to  2:1,  potentially  impacting  the  testing  results  of  the  product  parameter. 

The  supplier  may  apply  these  considerations  in  his  decisions  as  to  what  is 
"significantly"  out-of-tolerance,  whether  these  decisions  are  made  before  or  after  the  fact,  as 
described  in  the  preceding  paragraph.  For  example,  for  any  given  accuracy  ratio,  the  risk  of 
accepting  a  bad  calibration  or  of  rejecting  a  good  calibration,  can  be  evaluated  statistically. 

3.2.7  Adequacy  of  Calibration/Verification  System.  The  supplier  shall  establish  and 
maintain  documented  procedures  to  evaluate  the  adequacy  of  the  calibration/verification  system 
and  to  ensure  compliance  with  the  requirements  of  this  Standard. 

3. 2. 7.1  Interpretive  Guidance.  The  Standard  requires  that  the  supplier  provide  a  means  to 
monitor  and  evaluate  the  calibration/verification  system  against  the  requirements  of  this  Standard 
and  to  ensure  that  the  system  continues  to  fulfill  contractual  requirements. 

This  requirement  places  the  responsibility  on  the  supplier  to  perform  the  necessary 
quality  control/quality  assurance  functions  and  inspections  to  verify  that  all  aspects  of  the 
calibration/verification  system  are  in  compliance,  to  identify  and  evaluate  possible  and  actual 
calibration/verification  system  problems,  and  to  initiate  appropriate  safeguards  and  corrective 
actions. 

3. 2. 7. 2  Example(s).  The  supplier  has  established  procedures  including  audits,  inspections  or 
other  control  functions  for  assuring  that  all  aspects  of  the  calibration/verification  system  are  in 
compliance  with  the  Standard  specifically,  the  supplier  will  review  and  audit  operations  to  assure 
compliance  to  the  detailed  requirements  of  the  Standard  concerning  adequacy  of  measurement 
standards,  environmental  controls,  calibration  intervals,  calibration  procedures,  out-of-  tolerance 
reporting,  calibration  sources,  record  keeping,  calibration  labeling,  subcontractor  calibration,  and 
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storage  and  handling.  In  addition  to  these  specific  requirements,  the  supplier's  review  may 
include  related  elements  which  could  impact  operations  such  as  adequacy  of  M&TE,  personnel, 
and  training.  The  supplier  has  developed  a  system  wherein  all  reports  of  problems  or 
deficiencies  uncovered  in  system  audits  or  determined  during  normal  operations  are  documented, 
thoroughly  investigated  and  resolved. 

The  supplier  can  demonstrate  how  reviews  and  self-evaluations  of  the  calibration/verification 
system  are  performed.  Results  of  the  reviews  should  be  available.  The  supplier  may  wish  to  use 
the  in-house  quality  component  in  lieu  of  the  calibration  laboratory  in  performance  of  these 
audits  to  ensure  objectivity. 

3.2.8  Calibration  Sources.  M&TE  requiring  calibration  shall  be  calibrated  or  verified  by 
laboratories  meeting  the  requirements  of  2  of  this  Standard. 

3. 2. 8.1  Interpretive  Guidance.  See  2  of  the  Handbook. 

3. 2. 8.2  Example(s).  See  2  of  the  Handbook. 

3.2.9  Records.  The  requirements  of  this  Standard  shall  be  supported  by  records 
documenting  that  established  schedules  and  procedures  are  followed  to  maintain  the  adequacy  of 
all  M&TE.  The  records  for  M&TE  requiring  calibration  shall  include  an  individual  record  of 
calibration  or  verification,  or  other  means  of  control,  providing  a  description  or  identification  of 
the  item,  calibration  interval,  date  calibrated,  identification  of  the  calibration  source,  calibration 
results  (data  and/or  condition  status)  and  calibration  action  taken  (adjusted,  repaired,  new  value 
assigned,  derated,  etc.). 

3.2.9. 1  Interpretive  Guidance.  Records  provide  objective  evidence  that  the  laboratory  is 
adhering  to  calibration  schedules.  They  may  be  used  to  document  instrument  traceability. 
Records  also  provide  a  history  of  instrument  stability  and  reliability  which  may  be  evaluated  and 
used  as  a  basis  for  adjustment  of  calibration  intervals. 

The  Standard  does  not  prescribe  the  length  of  time  such  records  should  be  retained  by 
the  supplier.  However,  it  may  be  of  mutual  benefit  to  both  the  customer  and  the  supplier  to 
retain  the  records  as  long  as  the  M&TE  is  used  on  current  contracts. 
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3. 2. 9. 2  Excimple(s).  Adequate  calibration  records  are  kept  by  the  supplier  for  all  M&TE. 
Records  may  be  in  a  format  developed  at  the  supplier's  discretion,  but  must  be  kept  in  a  manner 
readily  available  for  use  by  authorized  personnel.  The  file  may  be  in  electronic  media,  such  as 
computer  records.  A  supplier's  record  system  readily  reveals  the  following  data  for  all  M&TE 
used  by  the  supplier  to  determine  compliance  to  contract  requirements: 

1)  Identification  of  each  piece  of  M&TE. 

2)  Current  calibration  interval  and  date  of  last  calibration. 

3)  Calibration  source  (laboratory). 

4)  Calibration  procedure  used. 

5)  Results  of  previous  calibrations  such  as  condition  found,  (in-tolerance  or  out-of¬ 
tolerance),  measured  value  if  a  fixed  measurement  standard,  and  any  limitations 
of  use. 

6)  Actions  taken  as  a  result  of  calibration  such  as  any  adjustments  or  corrections 
made,  any  repairs  performed  or  parts  replaced. 

7)  Any  pertinent  occurrences  not  directly  related  to  periodic  calibration,  such  as 
operational  failures  and  erratic  behavior. 

3.2.10  Calibration  Status.  M&TE  shall  be  labeled  to  indicate  calibration  or  verification 
status.  The  label  shall  identify  specific  date  calibrated  (day,  month,  year,  Julian  date,  or 
equivalent)  and  the  specific  calibration  due  date  or  usage  equivalent.  Items  not  calibrated  to  their 
full  capability  or  which  have  other  limitations  of  use,  shall  be  labeled  or  otherwise  identified  as 
to  the  limitations.  When  it  is  impractical  to  apply  a  label  directly  to  an  item,  the  label  may  be 
affixed  to  the  instrument  container  or  some  other  suitable  means  may  be  used  to  reflect 
calibration  status.  Tamper-resistant  seals  are  affixed  to  operator  accessible  controls  or 
adjustments  which  if  moved  will  invalidate  the  calibration.  The  supplier  shall  provide 
instructions  for  the  disposition  of  equipment  with  broken  tamper-resistant  seals. 

3.2.10.1  Interpretive  Guidance.  Each  piece  of  M&TE  should  be  labeled  to  provide  a  visual 
indication  of  its  calibration  status. 

The  label  indicates  date  last  calibrated,  date  due  for  calibration,  and  any  known 
limitations  of  use.  In  addition,  instruments  overdue  for  calibration  should  be  clearly  marked  to 
prevent  use.  While  some  suppliers  may  use  relatively  sophisticated  labeling  systems  (e.g.,  bar 
codes  and  light  beam  reading  devices)  to  provide  instrument  identification,  it  is  still  required  that 
the  calibration  status  (as  indicated  above)  be  visually  presented.  Operator  accessible  controls  or 
adjustments  that  could  invalidate  the  calibration  require  sealing  to  prevent  tampering. 

3.2.10.2  Example(s).  Each  piece  of  M&TE  used  by  the  supplier,  whether  supplier  owned  or 
leased,  customer  owned  or  leased,  or  personally  owned  by  the  supplier's  employees,  displays 
current  calibration  status,  in  terms  of  date  calibrated,  date  calibration  due,  and  any  use 
limitations.  The  method  of  expressing  calibration  dates  is  left  to  the  supplier;  however,  the  dates 
are  explicit.  When  labels  are  used  as  the  means  of  indicating  calibration  status,  the  labels  are 
distinctive  and  located  on  the  instrument  so  as  to  be  readily  visible. 

When  it  is  impractical  to  apply  labels  directly  to  the  instrument  (such  as  gage  blocks 
or  mass  standards),  the  label  is  applied  to  the  container  or  is  otherwise  accessible  to  the  user. 
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Some  instruments,  especially  measurement  standards  that  require  reference  to  a  certificate  or 
report  of  calibration  to  be  used  are  appropriately  labeled. 

The  supplier  has  affixed  tamper-resistant  seals  to  operator  accessible  controls  or 
adjustments  on  the  M&TE  which  if  moved  could  affect  the  calibration. 

Responsibility  for  identifying  the  instruments  requiring  such  seals  rests  with  the 
supplier  based  on  his  knowledge  of  how  the  instruments  are  used  in  support  of  the  contract. 

Such  items  should  be  removed  from  use  if  the  seals  are  broken  or  otherwise  compromised. 

The  supplier's  documented  calibration/verification  system  required  in  the  Standard 
contains  a  complete  description  of  the  calibration  status  labels  to  assure  understanding  on  the 
part  of  the  users  of  M&TE  and  the  calibration  laboratory.  The  use  of  the  tamper  resistant  seals 
on  operator  accessible  controls  and  adjustments,  and  disposition  of  M&TE  on  which  such  seals 
are  broken,  is  also  included  in  the  system  documentation.  When  M&TE  is  calibrated  by  other 
than  the  supplier,  and  different  labels  are  used,  the  labels  should  meet  the  requirements  of  the 
Standard  and  be  described  in  the  calibration/verification  system  description. 

3.2. 1 1  Control  of  Subcontractor  Calibration.  The  supplier  is  responsible  for  assuring  that  the 

subcontractors  calibration/verification  system  conforms  to  2  and  3  of  this  Standard  to  the  degree 
necessary  to  assure  compliance  with  contractual  requirements.  Accreditation  of  a  laboratory  to  2 
of  this  Standard  by  a  third  party  activity  acceptable  to  the  customer  may  serve  as  the  basis  for 
compliance  with  this  requirement. 

3.2.11.1  Interpretive  Guidance.  All  measurements  and  calibrations  performed  outside  the 
supplier's  plant,  which  may  affect  the  quality  or  compliance  with  requirements  of  supplies  or 
services  presented  to  the  Customer  for  acceptance,  should  be  subject  to  the  requirements  of  the 
Standard.  The  subcontractor  could  be  a  supplier  that  uses  M&TE  to  test  parts  or  products  to  be 
delivered  to  the  supplier,  or  a  calibration  or  standards  laboratory  providing  calibration  services 
for  the  supplier's  M&TE.  In  either  instance,  the  supplier  has  the  responsibility  to  impose  the 
applicable  requirements  and/or  to  assure  that  the  subcontractor  is  in  compliance  with  this 
Standard. 

3.2.11.2  Example(s).  The  supplier  has  imposed  appropriate  requirements  on  subcontractors 
(including  suppliers  and  vendors)  necessary  to  assure  compliance  with  this  Standard.  The 
strongest  requirement  by  the  supplier  would  be  incorporation  of  the  Standard  in  the  subcontract. 
A  lesser  requirement,  which  might  be  as  effective  depending  on  the  circumstances,  would  be 
incorporation  of  only  portions  of  the  Standard  in  the  contract  or  incorporation  of  comparable 
requirements  as  fitting. 

The  supplier  will  review  or  audit  the  subcontractor  to  assure  compliance  with  this 
Standard  or  whatever  specific  requirements  were  imposed.  Results  of  the  supplier's  reviews 
should  be  available. 

The  subcontractor,  especially  those  providing  calibration  services,  may  already  be 
subject  to  the  provisions  of  this  Standard  and  undergone  assessments  or  audits  by  an  accrediting 
or  other  customer  approved  body.  If  available  to  the  supplier,  the  results  of  such  assessments  or 
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audits  of  the  subcontractor  may  be  useful  in  assessing  compliance  to  the  supplier's  specific 
requirements.  However  for  previous  audits  or  assessments  to  be  meaningful,  they  should  have 
covered  the  same  or  very  similar  measurement  and  calibration  requirements  as  those  of  the 
supplier's.  Also,  the  previous  audit/assessment  findings  should  still  be  valid.  The  use  of 
previous  assessment  or  audit  result  does  not  relieve  the  supplier  of  the  responsibility  to  assure  the 
subcontractor's  compliance  to  requirements. 

3.2.12  Storage  and  Handling.  M&TE  shall  be  handled,  stored,  and  transported  in  a  manner 
which  shall  not  adversely  affect  the  calibration  or  condition  of  the  equipment. 

3.2.12.1  Interpretive  Guidance.  Proper  storage,  handling  and  transportation  of  M&TE  are 

essential  to  assure  that  required  accuracy  and  reliability  are  maintained. 
Vibration,  shock,  temperature  variances,  humidity  and  dust  are  only  some  of  the 
factors  that  could  have  an  adverse  impact  on  the  capability  of  the  instruments. 

The  supplier  must  describe  the  proper  methods  of  handling,  transporting,  and 
storing  of  M&TE  in  his  calibration/verification  system  description. 

3.2.12.2  Example(s).  A  procedure  for  handling,  transporting  and  storing  M&TE  is  in  place. 

The  following  factors  are  included  in  the  procedure  as  appropriate: 

1)  Equipment  is  appropriately  protected  prior  to  and  during  transportation. 

2)  Handling  of  equipment  during  transportation  or  use  follows  manufacturer's 
recommendations  and  good  commercial  practice. 

3)  Storage  areas  for  equipment  provide  acceptable  conditions  of  temperature, 
humidity,  and  other  pertinent  factors  to  preclude  deterioration. 

4)  The  supplier's  documented  procedures  require  that  instances  of  improper 
handling,  storage,  or  transportation  be  investigated  for  adverse  effects  on  the 
M&TE  and  appropriate  corrective  action  is  taken. 

4  PART  IV:  EXAMPLES 

4.1  Example  1:  Report  of  Calibration  and  Certificates 

See  Example  1. 

4.2  Example  2  :  Certificate  of  Calibration 

See  Example  2. 
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Report  of  Calibration 
(Example  1 ) 

Date  of  Issue:  May  1,  2007 


[Name  of  the  Calibration  Laboratories] 

[Address  of  the  calibration  laboratory] 

Calibration  Item 
Manufacturer:  _ 

Model  No.  _ 

Serial  No.  _ 

Control  No.  _ 

The  item  to  be  calibrated  was  received  in  good  condition,  in  tolerance,  and  required  no  special 
preparation  prior  to  measurement  in  this  laboratory. 

Measurements  were  made  using  [name  of  the  instrument(s)]  by  [name  of  the  methods  or 
procedure]  traceable  to  the  National  Institute  of  Standards  and  Technology  (NIST). 

The  reported  value(s)  and  uncertainties  resulting  from  the  measurement  process  are: 

[Parameter  calibrated]  [Unit]  [Value]  [Uncertainty]* 

*Coverage  factor  k=2 

Reference  temperature:  100.00  °C  ±  0.005  °C 
Room  ambient  conditions:  Temperature:  23  °C  ±  1  °C; 

Relative  humidity:  45%  RH  ±5%  RH 


Report  No.  [xxx-xxxxxxx-xx]  Page  1  of  3 


Submitted  By: 
Company: 
Address: 
Telephone: 
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Date  of  Issue:  May  1,  2007 


Equipment  and  Procedures  Used: 

Description  ID  Number  Exp./Issue  Date 

Blackbody,  M340  10025  12/20/07 

[Other  equipment  used]  10038  1/12/08 
[Calibration  procedures  used]  [xxx]  6/30/02 


Notes 

1)  Three  sets  of  ten  measurements  over  a  period  of  90  min  were  made  on  the  test 
[calibration  item]  on  each  alternate  day  between  [starting  date]  and  [ending  date]  at  a  nominal 
[parameter  calibrated]  of  [number]  [unit]  with  indicated  reference  temperature  of  100  °C. 

2)  The  [calibration  parameter]  value  reported  is  the  grand  average  of  the  three  sets  of  ten 

measurements  corrected  to . 

3)  The  uncertainty  reported  is  two  times  ( k=2 )  the  combined  standard  uncertainty  including 
the  random  variance  of  the  three  sets  of  measurements,  the  uncertainty  of  the  reference 
temperature,  and  those  of  the  [xxx]  used  in  the  calibration,  expressed  as  variances.  Other 
possible  sources  of  uncertainty  such  as  [xxx]  relative  to  ambient  temperature  were  negligible. 

4)  The  standards  and  calibration  program  of  the  [name  of  the  calibration  laboratories] 
complies  with  the  requirements  of  ANSI/MIL-STD-45662A/[This  Standard]  and  is  approved  by 
the  US  Calibration  Laboratory  Approval  Agency  (USCLAA)  for  the  parameter(s)  listed. 


Report  No.  [xxx-xxxxxxx-xx]  Page  2  of  3 
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Date  of  Issue:  May  1,  2007 
General  Conditions 


Unless  Otherwise  Stated: 

•  No  allowance  has  been  made  for  the  instability  of  the  test  device  due  to  use,  time,  etc., 
such  allowance  would  be  made  by  the  customer  as  needed. 

•  No  sampling  plan  or  other  sampling  process  was  used  for  this  calibration  and  the  result(s) 
reported  herein  apply  only  to  the  calibration  item  described  above. 

•  There  are  no  special  limitations  of  use  imposed  on  the  calibration  item. 

•  This  report  may  not  be  reproduced,  excepted  in  full,  unless  permission  for  the  publication 
of  an  approved  abstract  is  obtained  in  writing  from  the  calibrating  organization  issuing 
this  report. 


Measurement  Performed  by:  Approved  by: 

Name  Name: 

Title:  Metrology  Technician  Title:  Laboratory  Manger 

Cal.  Completion  Date:  Phone 


Report  No.  [xxx-xxxxxxx-xx]  Page  3  of  3 
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Certificate  of  Calibration 
(Example  2) 


Date  of  Issue:  May  1,  2007 


[Name  of  the  Calibration  Laboratories] 
[Address  of  the  calibration  laboratory] 


Calibration  Item: 
Manufacturer: 
Model  No. 

Serial  No. 
Control  No. 


Submitted  By: 
Company: 
Address: 
Telephone: 


Item  Condition/Special  Customer  Requirements 
As  Received  As  Left 

In  Tolerance*  M  In  Tolerance* 

I  J  Out  of  Tolerance*  l  ]  Out  of  Tolerance* 

1  J  Inoperative  l  ]  Inoperative 

Limited  Calibration**  Limited  Calibration** 

1  1  Adjusted 

*  Per  referenced  calibration  procedure. 

**  Limited  by  customer 

Remarks:  Only  [calibration  parameter]  calibrated  per  customer  request. 

Room  Ambient  Conditions:  Temperature  23  °C±1  °C,  relative  humidity  45%  RH  ±5%  RH. 
Procedure  Used:  xx-xxxxxx-x 


Certificate  No.  [xxx-xxxxxxx-xx]  Page  1  of  2 


7-63 


RCC  Special  Report:  Development  Of  Electro-Optical  (EO)  Standard  Processes  for  Application,  November  2011 


Date  of  Issue:  May  1,  2007 


Notes  /  General  Conditions: 

a.  The  standards  and  calibration  program  of  [name  of  the  calibration  laboratory ] 

complies  with  the  requirement  of  ANSI/MIL-STD-45662A/[This  Standard], 

b.  Unless  otherwise  stated  or  provided  for,  the  following  apply. 

(1)  Standards  used  in  this  calibration,  described  in  the  referenced  calibration 
procedure  with  associated  uncertainties  or  tolerances,  are  traceable  to  the  National 
Institute  of  Standards  and  Technology  (NIST). 

(2)  There  are  no  special  limitations  of  use  imposed  on  the  calibration  item. 

(3)  This  report  may  not  be  published,  except  in  full,  unless  permission  for  the 
publication  of  an  approved  abstract  is  obtained  in  writing  from  the  calibrating 
organization  issuing  this  report. 


Measurement  Performed  by:  Approved  by: 

Name  Name: 

Title:  Metrology  Technician  Title:  Laboratory  Manger 

Cal.  Completion  Date:  Phone 


Certificate  No.  [xxx-xxxxxxx-xx] 


Page  2  of  2 
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5  PART  V:  SAMPLE  CHECKLIST  FOR  EVALUATION  OF 
CALIBRATION  SYSTEMS 

The  sample  checklist  provided  on  the  following  pages  was  created  to  serve  as  an  aid 
for  the  evaluation  of  a  calibration  laboratory’s  compliance  with  the  Handbook  of  Electro-Optical 
Measurements ,  “ General  Requirements  for  the  Competence  of  Calibration  Laboratories" . 

The  checklist  is  keyed  to  the  EOSPA  for  guidance  only  and  should  not  be  viewed  as  a 
requirements  document.  It  is  not  intended  to  add  to,  or  subtract  from,  nor  in  anyway  to  modify 
the  EOSPA  standard  proper.  However,  other  items  may  be  added  based  on  the  scope  and/or 
quality  system  under  review. 

The  Control  Number  is  provided  to  facilitate  the  entry  of  a  unique  evaluation 
identifier,  e.g.,  Control  Number  123  ox  ABC.  The  Doc.  No.  is  provided  to  annotate  document 
number,  if  applicable,  of  the  documentation  describing  the  system  being  evaluated.  The 
Objective  Evidence  column  is  provided  to  allow  the  annotation  of  those  documents  which  may 
address  a  specific  area  of  the  EOSPA.  The  “ S/U/N ”  column  may  be  used  to  note  the  level  of 
compliance:  “S”  for  Satisfactory,  “IT”  for  Unsatisfactory,  “AT’  for  Needs  Improvement.  A  remark 
column  is  provided  in  the  checklist  for  the  annotation  of  comments  as  appropriate. 
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SAMPLE  CHECKLIST  FOR  EVALUATION  OF  CALIBRATION  SYSTEMS 

HANDBOOK  OF  ELECTRO-OPTICAL  MEASUREMENTS 

GENERAL  REQUIRMENTS  FOR  CALIBRATION  LABORATORIES 
AND  MEASURING  /  TEST  EQUIPMENT  (M&TE) 


Control  No.  _  Auditing  Agency 


Organization  _  Audit  Evaluator 


Calibration  System  Description: 

Doc.  No.  Audit  Date 


PART  II:  General  Requirements  for  the  Competence  of  Calibration  Laboratories. 

Chapter  II  of  the  Handbook  of  Electro-Optical  Standard  for  Applications  (EOSPA  )  applies  to  calibration  laboratories  in  the 
development  and  implementation  of  their  quality  systems. 


EOSPA  Evaluation  Factors 

Objective  Evidence 

S 

u 

N 

Remarks 

2.1  Laboratory  Organization  and  Management 

2.1.1  Legal  Identity 

The  laboratory  shall  be  legally  identifiable.  It  shall  be  organized 
and  shall  operate  in  such  a  way  that  its  permanent,  temporary, 
and  mobile  facilities  meet  the  requirements  of  EOSPA. 

2.1.2  Managerial  Staff 

The  laboratory  shall  have  managerial  staff  with  the  authority  and 
resources  needed  to  discharge  their  duties. 
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EOSPA  Evaluation  Factors 

Objective  Evidence 

S 

u 

N 

Remarks 

2.1.3  Personnel 

The  laboratory  shall  have  arrangements  to  ensure  that  its 
personnel  are  free  from  any  undue  pressures  which  might 
adversely  affect  the  quality  of  their  work. 

2.1.4  Organization  Structure 

The  laboratory  shall  be  organized  in  such  a  way  that  confidence 
in  its  independence  of  judgment  and  integrity  is  maintained  at 
all  times. 

Error!  Reference  source  not  found.Error!  Reference  source 
not  found.2.1.5  Error!  Reference  source  not  found.Error! 
Reference  source  not  found.  Documentation 

The  laboratory  shall  specify  and  document  the  responsibility, 
authority  and  interrelation  of  all  personnel  who  manage,  perform 
or  verify  work  affecting  the  quality  of  calibrations. 

2.1.6  Supervision 

The  laboratory  shall  provide  supervision  by  persons  familiar 
with  the  calibration  methods  and  procedures,  the  objective  of  the 
calibration  and  the  assessment  of  the  results.  Management 
practices  shall  be  such  as  to  ensure  adequate  supervision. 

2.1.7  Technical  Manager 

The  laboratory  shall  have  a  technical  manager  (however  named) 
who  has  overall  responsibility  for  the  technical  operations. 

2.1.8  Quality  Manager 

The  laboratory  shall  have  a  quality  manager  (however  named) 
who  has  responsibility  for  the  quality  system  and  its 
implementation.  The  quality  manager  shall  have  direct  access  to 
the  highest  level  of  management  at  which  decisions  are  taken  on 
laboratory  policy  or  resources,  and  to  the  technical  manager.  In 
some  laboratories,  the  quality  manager  may  also  be  the  technical 
manager  or  deputy  technical  manager. 
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EOSPA  Evaluation  Factors 

Objective  Evidence 

S 

u 

N 

Remarks 

2.1.9  Designated  Alternates 

The  laboratory  shall  designate  alternates  in  case  of  absence  of 
the  technical  or  quality  manager. 

2.1.10  Policy  and  Procedures 

The  laboratory  shall  have,  where  relevant,  have  documented 
policy  and  procedures  to  ensure  the  protection  of  customer's 
confidential  information  and  proprietary  rights. 

2.1.11  Measurement  Assurance  Programs 

The  laboratory  shall,  where  appropriate,  participate  in  inter 
laboratory  comparisons  and  proficiency  testing  programs. 

2.2  Quality  System,  Audit,  and  Review 

2.2.1  Quality  System 

The  laboratory  shall  establish  and  maintain  a  quality  system 
appropriate  to  the  type,  range  and  volume  of  calibration 
activities  it  undertakes.  The  elements  of  this  system  shall  be 
documented.  The  quality  documentation  shall  be  available  for 
use  by  the  laboratory  personnel.  The  laboratory  shall  define  and 
document  its  policies  and  objectives  for,  and  its  commitment  to, 
good  laboratory  practice  and  quality  of  calibration  services.  The 
laboratory  management  shall  ensure  that  these  policies  and 
objectives  are  documented  in  a  quality  manual  and 
communicated  to,  understood,  and  implemented  by  all 
laboratory  personnel  concerned.  The  quality  manual  shall  be 
maintained  current  under  the  responsibility  of  the  quality 
manager. 

2.2.2  Quality  Manual 

The  quality  manual  and  related  documentation  shall  state  the 
laboratory's  policies  and  operational  procedures  established  in 
order  to  meet  the  requirements  of  this  Standard. 

2.2.2. 1-1  A  quality  policy  statement,  including  objectives,  by 
top  management 
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2.2.2. 1-2  The  organization  and  management  structure  of  the 
laboratory,  its  place  in  any  parent  organization,  related 
organization  charts 

2.2.2. 1-3  The  relations  between  management,  technical 
operations,  support  services  and  the  quality  system 

2.2.2. 1-4  Procedures  for  control  and  maintenance  of 
documentation 

2.2.2. 1-5  Job  descriptions  of  key  staff  and  reference  to 
descriptions  of  other  staff 

2.2.2. 1-6  Identification  of  the  approved  signatories  of  the 
laboratory 

2.2.2. 1-7  The  laboratory's  procedures  for  achieving  traceability 
of  measurements 

2.2.2. 1-8  The  laboratory's  scope  of  calibrations  and/or 
verifications 

2.2.2. 1-9  Arrangements  for  ensuring  that  the  laboratory  reviews 
all  new  work  to  ensure  that  it  has  the  appropriate  facilities  and 
resources  before  commencing  such  work 

2.2.2.1-10  Reference  to  the  calibration  and  verification 
procedures  used 

2.2.2. 1- 1 1  Procedures  for  handling  calibration  and  verification 
items 

2.2.2.1-12  Reference  to  the  major  equipment  and  reference 
measurement  standards  used 

2.2.2.1-13  Reference  to  procedures  for  calibration,  verification 
and  maintenance  of  equipment  used 

2.2.2.1-14  Reference  to  quality  assurance  practices  including 
inter  laboratory  comparisons,  proficiency  testing  programs,  use 
of  reference  materials,  and  internal  quality  control  schemes 
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2.2.2.1-15  Procedures  to  be  followed  for  feedback  and 
corrective  action  whenever  measurement  discrepancies  are 
detected,  or  departures  from  documented  policies  and 
procedures  occur 

2.2.2.1-16  The  laboratory  management  arrangements  for 
exceptionally  permitting  departures  from  documented  policies 
and  procedures  or  from  standard  specifications 

2.2.2.1-17  Procedures  for  dealing  with  complaints 

2.2.2.1-18  Procedures  for  protecting  confidentiality  and 
proprietary  rights 

2.2.2.1-19  Procedures  for  audit  and  review 

2.2.2.1-20  A  statement  of  the  lab's  policy  for  establishing  and 
changing  calibration  intervals  for  equipment  it  controls 

2.2.2.1-21  A  statement  of  the  laboratory's  policy  concerning  the 
technique(s)  to  be  used  for  determining  measurement 
uncertainty  and  calibration/verification  adequacy 

2.2.3  Quality  Audit 

The  laboratory  shall  arrange  for  audits  of  its  activities  at 
appropriate  intervals  to  verify  that  its  operations  continue  to 
comply  with  the  requirements  of  the  quality  system.  Such  audits 
shall  be  carried  out,  wherever  possible,  by  trained  and  qualified 
staff  who  are  independent  of  the  activity  to  be  audited.  Where 
the  audit  findings  cast  doubt  on  the  correctness  of  validity  of  the 
laboratory's  calibrations  results,  the  laboratory  shall  take 
immediate  corrective  action  and  shall  immediately  notify,  in 
writing,  any  client  whose  work  may  have  been  affected. 
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2.2.4  Quality  System  Review 

The  quality  system  adopted  to  satisfy  the  requirements  of  this 
Standard  shall  be  reviewed  at  least  once  a  year  by  the 
management  to  ensure  its  continuing  suitability  and 
effectiveness  and  to  introduce  any  necessary  changes  or 
improvements. 

2.2.5  Documentation 

All  audit  and  review  findings  and  any  corrective  actions  that 
arise  from  them  shall  be  documented.  The  person  responsible 
for  quality  shall  ensure  that  these  actions  are  discharged  within 
the  agreed  timeframe. 

2.2.6  Quality  Check 

In  addition  to  periodic  audits  the  laboratory  shall  ensure  the 
quality  of  results  provided  to  customers  by  implementing 
checks.  These  checks  shall  be  reviewed  and  shall  include,  as 
appropriate,  but  not  be  limited  to: 

2.2.6. 1  internal  quality  control  using,  wherever  possible, 
statistical  techniques 

2. 2. 6. 2  participation  in  proficiency  testing  or  other  inter 
laboratory  comparisons 

2. 2. 6. 3  regular  use  of  certified  reference  materials  and/or  in- 
house  quality  control  using  reference  materials 

2. 2. 6.4  replicate  measurements  using  the  same  or  different 
methods 

2. 2. 6. 5  correlation  of  results  for  different  characteristics  of  an 
item 
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2.3  Personnel 

2.3.1  Staff  Size 

The  calibration  laboratory  shall  have  sufficient  personnel  with 
the  necessary  education,  training,  technical  knowledge  and 
experience  for  their  assigned  functions. 

2.3.2  Staff  Training 

The  calibration  laboratory  shall  ensure  that  the  training  of  its 
personnel  is  kept  up-to-date  consistent  with  employee 
assignments  and  development. 

2.3.3  Staff  Records 

Records  on  the  relevant  qualifications,  training,  skills  and 
experience  of  the  technical  personnel  shall  be  maintained  and  be 
available  to  the  laboratory. 

2.4  Accommodation  and  Environment 

2.4.1  Laboratory  Environmental  Conditions 

Laboratory  accommodation  (facilities),  calibration  areas,  energy 
sources,  lighting,  temperature,  humidity,  and  ventilation  shall  be 
such  as  to  facilitate  proper  performance  of 
calibrations/verifications . 

2.4.2  Experiment  Environmental  Conditions 

The  environment  in  which  these  activities  are  undertaken  shall 
be  specified  and  not  invalidate  the  results  or  adversely  affect  the 
required  uncertainty  of  measurement.  Particular  care  shall  be 
taken  when  such  activities  are  undertaken  at  sites  other  than  the 
permanent  laboratory  premises. 

2.4.3  Monitor,  Control,  and  Record 

The  laboratory  shall  effectively  monitor,  control  and  record 
environmental  conditions  as  appropriate.  Due  attention  shall  be 
paid,  for  example,  to  biological  sterility,  dust,  electromagnetic 
interference,  humidity,  line  voltage,  temperature,  and  sound  and 
vibration  levels,  as  appropriate  to  the  calibrations  concerned. 

7-73 


RCC  Special  Report:  Development  Of  Electro-Optical  (EO)  Standard  Processes  for  Application,  November  2011 


EOSPA  Evaluation  Factors 

Objective  Evidence 

S 

u 

N 

Remarks 

2.4.4  Sub  Zoning 

There  shall  be  effective  separation  between  neighboring  areas 
when  the  activities  therein  are  incompatible. 

2.4.5  Access  Control 

Access  to  and  use  of  all  areas  affecting  the  quality  of  these 
activities  shall  be  defined  and  controlled. 

2.4.6  Housekeeping 

Adequate  measures  shall  be  taken  to  ensure  good  housekeeping 
in  the  laboratory. 

2.5  Equipment  and  Reference  Materials 

2.5.1  Equipment  Availability 

The  laboratory  shall  be  furnished  with  all  items  of  equipment 
(including  reference  materials)  required  for  the  correct 
performance  of  calibrations/verifications.  In  those  cases  where 
the  laboratory  needs  to  use  equipment  outside  its  permanent 
control,  it  shall  ensure  that  the  relevant  requirements  of  this 
Standard  are  met. 

2.5.2  Equipment  Maintenance 

All  equipment  shall  be  properly  maintained.  Maintenance 
procedures  shall  be  documented.  Any  item  of  the  equipment 
which  has  been  subjected  to  overloading  or  mishandling,  or 
which  gives  suspect  results,  or  has  been  shown  by  verification  or 
otherwise  to  be  defective,  shall  be  taken  out  of  service,  clearly 
identified  and  wherever  possible  stored  at  a  specified  place  until 
it  has  been  repaired  and  shown  by  calibration  or  verification  to 
perform  satisfactorily.  The  laboratory  shall  examine  the  effect 
of  this  defect  on  previous  calibrations. 

2.5.3  Equipment  Labels 

Each  item  of  equipment  including  reference  materials  shall, 
when  appropriate,  be  labeled,  marked  or  otherwise  identified  to 
indicate  its  calibration  status. 
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2.5.4  Equipment  Records 

Records  shall  be  maintained  of  each  item  of  equipment  and  all 
reference  materials  significant  to  the  calibrations/verifications 
performed. 

2.5.4a.  The  name  of  the  item  of  equipment. 

2.5.4b.  The  manufacturer's  name,  type  identification,  and  serial 
number  or  other  unique  identification. 

2.5.4c.  Current  location,  where  appropriate. 

2.5.4dd.  Where  applicable,  dates  and  results  of  calibration 
and/or  verifications  and  date  or  criteria  when  the  calibration 
and/or  verification  expires. 

2.5. 4e.  Details  of  maintenance  carried  out  to  date  and  planned 
for  the  future. 

2.5. 4f.  History  of  any  damage,  malfunction,  modification  or 
repair. 

2.5. 4g.  Measured  value  observed  for  each  parameter  found  to 
be  out  of  tolerance  during  calibration/verification. 

2.6  Measurement  Traceability  and  Calibration 

2.6.1  Calibration  System 

All  measuring/testing  equipment  having  an  effect  on  the 
accuracy  or  validity  of  calibrations  shall  be  calibrated  and/or 
verified  before  being  put  into  service.  The  laboratory  shall  have 
an  established  program  for  the  calibration  and  verification  of  its 
measuring  and  test  equipment  to  ensure  the  recall  or  removal 
from  service  of  any  standard  or  equipment  which  has  exceeded 
its  calibration  interval  or  is  otherwise  judged  to  be  unreliable. 
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2.6.2  Calibration  Traceability 

The  overall  program  of  calibration  and/or  verification  of 
equipment  shall  be  designed  and  operated  so  as  to  ensure  that, 
wherever  applicable,  measurements  made  by  the  laboratory  are 
traceable  to  national,  international,  or  intrinsic  standards  of 
measurements  where  available.  Calibration  certificate  and/or 
report  shall,  wherever  applicable,  state  the  traceability  to 
national,  international,  or  intrinsic  standards  of  measurements 
and  shall  provide  the  measurement  results  and  associated 
uncertainty  of  measurement  and/or  a  statement  of  compliance 
with  an  identified  metrological  specification. 

2.6.3  Alternatives 

Where  traceability  to  international,  national,  or  intrinsic 
standards  of  measurement  is  not  available,  traceability 
requirements  may  be  satisfied  by: 

2.6.3a.  Participation  in  a  suitable  program  of  interlaboratory 
comparisons  or  proficiency  testing. 

2.6.3b.  Internationally  accepted  standards  in  the  field  concerned. 

2.6.3c.  Suitable  reference  materials. 

2.6.3d.  Ratio  or  reciprocity-type  measurements 

2.6.3e.  Mutual  consent  standards  which  are  clearly  specified  and 
mutually  agreed  upon  by  all  parties  concerned. 

2.6.4  Reference  Standards  Usage 

Reference  standards  of  measurements  held  by  the  laboratory 
shall  be  used  for  calibration  or  verification  only  and  for  no  other 
purpose,  unless  it  can  be  demonstrated  that  their  performance  as 
reference  standards  has  not  been  invalidated. 
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2.6.5  Reference  Standards  Calibration 

Reference  standards  of  measurement  shall  be  calibrated  by  a 
competent  body  that  can  provide  traceability.  There  shall  be  a 
program  of  calibration  and  verification  for  reference  standards. 

2.6.6  Additional  Validation 

Where  relevant,  reference  standards,  measuring  and  test 
equipment  shall  be  subject  to  in-service  checks  between 
calibrations  and  verifications. 

2.6.7  Reference  Material 

Reference  materials  shall,  where  possible,  be  traceable  to 
national  or  international  standards  of  measurements,  or  to 
national  or  international  standard  reference  materials. 

2.7  Calibration  Methods  and  Procedures 

2.7.1  Documentation 

The  laboratory  shall  have  documented  instructions  on  the  use 
and  operation  of  all  relevant  equipment,  on  the  handling  and 
preparation  of  items,  and  for  calibrations/verifications,  where 
the  absence  of  such  instructions  could  jeopardize  the 
calibrations/verifications.  All  instructions,  standards,  manuals, 
and  reference  data  relevant  to  the  work  of  the  laboratory  shall  be 
maintained  up-to-date  and  be  readily  available  to  the  staff. 

2.7.2  Use  of  Calibration  Methods  and  Procedures 

The  laboratory  shall  use  appropriate  methods  and  procedures  for 
all  calibrations/verifications  and  related  activities  within  its 
responsibility  (including,  but  not  limited  to,  sampling,  handling, 
transport  and  storage,  preparation  of  items,  estimation  of 
uncertainty  of  measurement  and  analysis  of  calibration  data). 
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2.7.2a  Calibration  procedures  shall  contain  the  required  range 
and  tolerance  or  uncertainty  of  each  item  or  unit  parameter  being 
calibrated  or  verified.  In  addition,  the  procedures  shall  contain 
the  generic  description  of  the  measurement  standards  and 
equipment  needed  with  the  required  parameter,  range,  tolerances 
or  uncertainties,  and  specifications  for  performing  the 
measurement  of  the  calibration  or  verification,  and/or 
representative  types  (manufacturer,  model,  option)  that  are 
capable  of  meeting  the  generic  description  for  the  measurement 
standards.  The  procedures  shall  be  consistent  with  the  accuracy 
required,  and  with  any  standard  specifications  relevant  to  the 
calibrations/verifications  concerned. 

2.7.2b  The  laboratory  shall  ensure  that  the  calibration 
uncertainties  are  sufficiently  small  so  that  the  adequacy  of  the 
measurement  is  not  affected.  Well  defined  and  documented 
measurement  assurance  techniques  or  uncertainty  analyses  may 
be  used  to  verify  the  adequacy  of  a  measurement  process.  If 
such  techniques  or  analyses  are  not  used,  then  the  collective 
uncertainty  of  the  measurement  standards  shall  not  exceed  25% 
of  the  acceptable  tolerance  (e.g.,  manufacturer's  specification) 
for  each  characteristic  of  the  measuring  and  test  equipment 
being  calibrated  or  verified. 

2.7.3  Standard  Calibration  Methods  and  Procedures. 

Where  methods  are  not  specified,  the  laboratory  shall,  wherever 
practical,  select  methods  that  have  been  published  in 
international  or  national  standards,  those  published  by  reputable 
technical  organizations  or  in  relevant  scientific  texts  or  journals. 
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2.7.4  Alternatives 

Where  it  is  necessary  to  employ  methods  that  have  not  been  well 
established,  these  shall  be  subject  to  agreement  with  the 
customer,  be  fully  documented  and  validated,  and  be  available 
to  the  customer  or  other  recipients  of  the  relevant  reports. 

2.7.5  Sampling  Methods 

Where  sampling  is  carried  out  as  part  of  the  calibration  method, 
the  laboratory  shall  use  documented  procedures  and  appropriate 
statistical  techniques  to  select  the  samples. 

2.7.6  Calculations  and  Data  Transfers 

Calculations  and  data  transfers  shall  be  subject  to  appropriate 
checks. 

2.7.7  Data  Automation 

Where  computers  or  automated  equipment  are  used  for  the 
capture,  processing,  manipulation,  recording,  reporting,  storage 
or  retrieval  of  calibration  data,  the  laboratory  shall  ensure  that: 

2.7.7a 

a.  The  requirements  of  this  Standard  are  followed. 

2.7.7b  b.  Computer  software  is  documented  and  adequate 

for  use. 

2.7.7c  c.  Procedures  are  established  and  implemented  for 

protecting  the  integrity  of  data.  Such  procedures  shall  include, 
but  are  not  limited  to,  integrity  of  data  entry  or  capture,  data 
storage,  data  transmission  and  data  processing. 

2.7. 7d4)  Computers  and  peripheral  equipment  used  for 

calibration  processes  have  recommended  environmental 
operating  needs  and  routine  maintenance  procedures  that  ensure 
their  continued  operation. 
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2.1. le.  It  establishes  and  implements  appropriate  procedures  for 
the  maintenance  of  security  of  data  including  the  prevention  of 
unauthorized  access  to,  and  the  unauthorized  amendment  of, 
computer  records. 

2.7.8  Consumable  Materials 

Documented  procedures  shall  exist  for  the  purchase,  reception 
and  storage  of  consumable  materials  used  for  the  technical 
operations  of  the  laboratory  that  can  affect  the  results  of 
calibrations. 

2.8  Handling  of  Calibration  Items 

2.8.1  Documentation 

The  laboratory  shall  have  a  documented  system  for  uniquely 
identifying  the  items  to  be  calibrated,  to  ensure  that  there  can  be 
no  confusion  regarding  the  identity  of  such  items  at  any  time. 

2.8.2  Records 

Upon  receipt  of  the  calibration  item,  any  abnormalities  or 
departures  from  standard  condition  as  prescribed  in  the  relevant 
calibration  method  shall  be  recorded.  Where  there  is  any  doubt 
as  to  the  item's  suitability  for  calibration,  where  the  item  does 
not  conform  to  the  description  provided,  or  where  the  calibration 
required  is  not  fully  specified,  the  laboratory  shall  consult  the 
customer  for  further  instruction  before  proceeding.  The 
laboratory  shall  establish  whether  the  item  has  received  all 
necessary  preparation,  or  whether  the  customer  requires 
preparation  to  be  undertaken  or  arranged  by  the  laboratory. 

2.8.3  Handling 

The  laboratory  shall  have  documented  procedures  and 
appropriate  facilities  to  avoid  deterioration  or  damage  to  the 
calibration  item,  during  storage,  handling,  preparation,  and 
calibration;  any  relevant  instructions  provided  with  the  item 
shall  be  followed.  Where  items  have  to  be  stored  or  conditioned 
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under  specific  environmental  conditions,  these  conditions  shall 
be  maintained,  monitored  and  recorded  where  necessary.  Where 
a  calibration  item  or  portion  of  an  item  is  to  be  held  secure  (for 
example,  for  reasons  of  record,  safety  or  value,  or  to  enable 
check  calibrations  to  be  performed  later),  the  laboratory  shall 
have  storage  and  security  arrangements  that  protect  the  condition 
and  integrity  of  the  secured  items  or  portions  concerned. 

2.8.4  Safety 

The  laboratory  shall  have  documented  procedures  for  the 
receipt,  retention  or  safe  disposal  of  calibration  items,  including 
all  provisions  necessary  to  protect  the  integrity  of  the  laboratory. 

2.8.5  Seals 

Tamper-resistant  seals  shall  be  affixed  to  operator  accessible 
controls  or  adjustments  on  measurement  standards  or  measuring 
and  test  equipment  which,  if  moved,  will  invalidate  the 
calibration.  The  laboratory's  calibration  system  shall  provide 
instructions  for  the  use  of  such  seals  and  for  the  disposition  of 
equipment  with  damaged  or  broken  seals. 

2.9  Records 

2.9.1  Laboratory  Records 

The  laboratory  shall  maintain  a  record  system  to  suit  its 
particular  circumstances  and  comply  with  any  applicable 
regulations.  The  records  for  each  calibration  shall  contain 
sufficient  information  to  permit  the  repetition  of  the  calibration. 
The  records  shall  include  the  identity  of  personnel  involved  in 
preparation  and  calibration. 

2.9.2  Bookkeeping 

All  records  (including  those  pertaining  to  calibration 
equipment),  certificates,  and  reports  shall  be  safely  stored  and 
held  secure  and  in  confidence  to  the  customer  for  the  period 
specified  in  the  quality  manual. 
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****  NOTHING  FOLLOWS  **** 


